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FOREWORD 


The  U.S.  Army  Research  Institute  Aviation  Research  and  De¬ 
velopment  Activity  (ARIARDA)  at  Fort  Rucker,  Alabama,  contributes 
to  the  effectiveness  of  Army  aviation  by  conducting  a  comprehen¬ 
sive  human  factors  research  program  in  support  of  aircrew  perfor¬ 
mance  and  training.  The  ARIARDA  research  program  encompasses  the 
full  scope  of  army  aviation,  with  projects  in  support  of  emerging 
Army  aviation  weapon  systems,  aviation  manpower  and  personnel 
programs,  aviator  training  programs,  and  aviator  safety  programs. 

This  report  summarizes  research  performed  and  products  de¬ 
veloped  in  all  four  of  the  above  areas  between  9  October  1988  ard 
8  October  1989  by  Anacapa  Sciences,  Inc.,  under  contract  to  the 
U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sci¬ 
ences.  Thirty-one  projects  are  summarized:  12  describe  research 
in  support  of  manpower  and  personnel  programs;  13  report  accom¬ 
plishments  in  support  of  aviator  training  programs;  and  1  de¬ 
scribes  research  in  aviation  safety.  In  addition,  four  technical 
advisory  services  are  described.  The  projects  in  emerging  sys¬ 
tems,  aviation  safety,  and  two  technical  advisory  services  are 
conducted  within  the  mission  of  the  Systems  Research  Laboratory 
at  the  Army  Research  Institute  (ARI) .  The  projects  in  manpower 
and  personnel  are  conducted  within  the  mission  of  the  Manpower 
and  Personnel  Research  Laboratory  at  ARI.  Finally,  the  aviator 
training  projects  and  two  technical  advisory  services  are  con¬ 
ducted  within  the  mission  of  the  Training  Research  Laboratory  at 
ARI.  Specific  taskings  are  identified  for  each  project  or  re¬ 
search  area,  and  the  utilization  of  the  research  findings  or 
products  is  described  in  each  summary  report. 

This  summary  report  is  designed  to  meet  two  important  ob¬ 
jectives.  First,  it  provides  a  summary  of  research  progress  and 
accomplishments  to  U.S.  Army  weapon  system  managers,  manpower  and 
personnel  planners,  and  training  system  developers  and  managers 
in  their  respective  areas  of  responsibility.  Second,  it  provides 
summary  information  to  behavioral  scientists  working  on  similar 
applied  research  issues,  either  in  the  Department  of  Defense  or 
in  other  governmental,  industrial,  or  university  organizations. 


EDGAR  M.  JOHNSON 
Technical  Director 
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EXECUTIVE  SUMMARY 


Requirement: 

Anacapa  Sciences,  Inc.,  has  provided  collocated  research 
support  to  the  U.S.  Army  Research  Institute  Aviation  Research  and 
Development  Activity  (ARIARDA)  at  Fort  Rucker,  Alabama,  since 
1981.  The  ARIARDA  program  supports  the  full  range  of  Army  avia¬ 
tion  research  requirements  with  projects  that  address  issues  in 
emerging  aviation  weapon  systems,  aviation  manpower  and  person¬ 
nel,  aviator  training,  and  aviation  safety.  This  Annual  Summary 
Report  fulfills  one  of  the  requirements  of  Contract  No.  MDA903- 
87-C-0523 .  It  describes  the  31  research  projects  and  4  technical 
advisory  services  conducted  by  Anacapa  Sciences,  Inc.,  research¬ 
ers  between  October  1988  and  October  1989  in  support  of  the 
ARIARDA  program.  The  specific  requirements  that  led  to  the  ini¬ 
tiation  of  each  research  project  are  discussed  in  the  individual 
summaries . 


Procedures : 

There  are  substantial  differences  in  the  methods  that  were 
employed  in  the  individual  projects  and  in  the  technical  advisory 
services.  In  some  cases,  the  research  approach  was  a  scientific 
experiment  in  which  selected  variables  were  controlled,  manipu¬ 
lated,  and  measured.  In  other  cases,  the  research  approach  was  a 
set  of  analytical  or  product  development  tasks.  The  specific 
research  methods  used  in  each  project  and  technical  advisory 
service  are  described  in  the  individual  summaries. 


Summary  of  Contents: 

The  research  projects  were  conducted  in  all  four  domains  of 
the  ARIARDA  research  program.  Twelve  of  the  projects  are  in  the 
emerging  aviation  weapon  systems  domain.  Ten  of  these  projects 
address  the  prediction  of  operator  workload  in  varying  configura¬ 
tions  of  the  LHX,  AH-64 ,  UH-60,  MH-60K,  CH-47,  and  MH-47E  air¬ 
craft.  The  other  two  projects  are  concerned  with  the  integration 
of  maintenance  considerations  during  the  early  design  phases  of 
new  aircraft  and  the  design  of  flight  symbology. 

Five  of  the  projects  are  in  the  manpower  and  personnel  re¬ 
search  domain:  an  evaluation  of  the  First  Army  Reserve  aviation 
management  method,  the  development  and  validation  of  a  new  avi¬ 
ator  selection  test  (two  projects) ,  the  development  of  a  peer 


vii 


assessment  method,  and  a  survey  of  aviation  ammunition  and  gun¬ 
nery  training  practices  and  requirements.  Thirteen  projects  are 
part  of  the  aviator  training  research  domain.  Six  of  these 
projects  are  concerned  with  the  evaluation  of  flight  simulator 
training;  the  other  seven  projects  are  concerned  with  developing 
an  AH-64  symbology  training  program,  upgrading  the  basic  map 
interpretation  and  terrain  analysis  course  to  videodisc,  survey¬ 
ing  the  research  on  computer-based  instruction,  and  conducting 
four  evaluations  of  the  effectiveness  of  aviation  part-task 
trainers.  Finally,  one  project  was  initiated  during  the  current 
contract  year  to  develop  and  evaluate  simulated  accident  scenario 
training  as  a  method  of  improving  aviation  safety. 

The  four  technical  advisory  services  were  concerned  with 
participating  in  the  Special  Operations  Aircraft  Program  in¬ 
progress  reviews  and  crew  station  working  group  meetings,  sup¬ 
porting  research  in  operator  workload  measurement,  developing 
software  for  a  flightline  research  system,  and  developing  and 
evaluating  a  computer-based  threat  training  program. 


Utilization  of  Findings; 

The  results  and  recommendations  of  many  of  the  projects  and 
technical  advisory  services  will  be  directly  implemented  in  the 
design  of  new  .aviation  systems,  in  the  selection  and  management 
of  aviation  personnel,  and  in  aviation  training  at  the  Aviation 
Center,  Fort  Rucker,  Alabama,  and  in  Army  aviation  units  around 
the  world.  This  report  provides  Army  weapon  systems  managers, 
manpower  and  personnel  planners,  training  system  developers  and 
managers,  and  other  researchers  working  in  related  fields  with  a 
summary  of  the  research  activities  in  their  respective  areas  of 
interest. 
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Introduction 

Anacapa  Sciences,  Inc.,  has  provided  collocated  research 
support  to  the  U.S.  Army  Research  Institute  Aviation  Research 
and  Development  Activity  (ARIARDA)  at  Fort  Rucker,  Alabama, 
under  a  series  of  contracts  that  began  1  September  1981.  The 
current  contract  (No.  MDA903-87-C-0523)  requires  the  submis¬ 
sion  of  an  Annual  Summary  Report  of  research  project  activi¬ 
ties.  This  report  describes  the  Anacapa  research  project 
activities  and  achievements  during  the  period  from  9  October 
1988  to  8  October  1989.  Throughout  the  report,  this  period  is 
referred  to  as  the  current  contract  year. 

Ten  of  the  research  summaries  in  this  report  describe 
individual  projects  that  Anacapa  Sciences  personnel  have 
worked  on  during  the  current  contract  year.  Eight  of  the 
summaries  describe  major,  long-term  research  areas  that  are 
divided  into  21  discrete  projects;  each  project  will  conclude 
with  a  deliverable  product.  The  projects  are  listed  in  the 
table  of  contents  under  the  title  of  the  research  area. 
Finally,  four  summaries  describe  technical  advisory  services, 
which  are  research  activities  or  expert  assistance  provided 
to  Army  programs  and  projects  that  are  not  directly  assigned 
to  Anacapa  Sciences. 

Most  of  the  project  and  technical  advisory  service 
summaries  follow  the  same  general  format.  Each  summary 
begins  with  a  background  section  that  presents  information 
needed  to  understand  the  requirement  for  the  project .  The 
background  may  include  a  brief  review  of  the  relevant 
research  literature  or  describe  the  critical  events  that  led 
to  the  initiation  of  the  project  or  technical  advisory 
service.  Where  appropriate,  the  relationship  between 
specific  projects  in  a  research  area  or  between  an  Anacapa 
research  project  and  a  technical  advisory  service  is 
discussed. 

When  the  need  for  the  research  cannot  be  clearly 
inferred  from  the  background  information,  a  statement  of  need 
or  definition  of  the  research  problem  is  presented.  This  is 
followed  by  a  concise  statement  of  the  project  or  research 
area  objectives.  Next,  the  research  approach  section 
presents  a  description  of  the  activities  that  were  planned  to 
accomplish  the  research  objectives.  For  some  projects,  the 
research  approach  is  a  scientific  experiment  in  which 
selected  variables  are  controlled,  manipulated,  and  measured. 
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For  other  projects,  the  research  approach  is  a  set  of 
analytical  or  product  development  tasks. 

The  research  approach  is  usually  followed  by  one  or  more 
sections  that  describe  the  work  completed  on  the  project  and 
research  findings  or,  in  the  case  of  product  development 
efforts,  a  summary  description  of  the  research  products.  In 
the  technical  advisory  service  summaries,  the  research 
approach  is  usually  followed  by  a  description  of  the  services 
provided  by  Anacapa  personnel.  The  final  section  of  each 
summary  describes  the  work  projected,  if  any.  Where 
possible,  this  section  also  presents  the  current  project 
milestones . 

Anacapa  personnel  also  provided  temporary  research, 
technical,  administrative,  and  logistical  support  on  other 
projects  that  are  the  primary  responsibility  of  ARIARDA 
personnel  and  are,  consequently,  not  summarized  in  this 
report.  It  is  important  to  note  that  the  projects  summarized 
in  this  report  represent  only  a  portion  of  ARIARDA' s  research 
program.  Numerous  other  projects  are  being  conducted  either 
in-house  by  ARIARDA  personnel  or  under  other  contracts. 

The  project  summaries  are  presented  in  four  content 
categories  that  reflect  the  research  domains  at  ARIARDA. 

This  organization  is  intended  to  assist  the  reader  in 
locating  a  specific  project  summary  within  a  research  domain 
or  in  finding  summaries  that  are  closely  related  in  terms  of 
content . 

The  first  six  summaries  describe  12  projects  in  emerging 
aviation  systems  design.  The  next  four  summaries  describe  5 
projects  in  manpower  and  personnel  research.  The  next  seven 
summaries  describe  13  aviator  training  research  projects. 

The  last  summary  describes  a  new  project  designed  to  develop 
and  evaluate  simulated  accident  scenario  training  as  a  method 
of  increasing  aviator  safety.  The  number  of  projects 
assigned  to  the  four  categories  is  not  necessarily  in 
proportion  to  the  emphasis  placed  on  each  research  domain. 

The  project  summaries  are  followed  by  descriptions  of 
the  four  technical  advisory  services.  Technical  advisory 
services  were  provided  to  support  (a)  aviation  system 
research  and  development  programs  being  managed  by  Aviation 
Systems  Command  managers  and  engineers  and  (b)  aviation 
projects  being  conducted  by  ARIARDA  scientists.  The  first 
two  summaries  describe  technical  advisory  services  provided 
in  support  of  emerging  aviation  systems  design.  The  third 
summary  describes  support  provided  in  the  development  of  a 
flightline  research  data  management  system.  The  fourth 
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summary  describes  a  technical  advisory  service  provided  in 
support  of  an  ARIARDA  training  research  project. 

Although  each  summary  identifies  the  project  director  or 
technical  advisor (s),  the  Anacapa  approach  to  research 
employs  a  team  concept.  This  approach  provides  the  optimum 
utilization  of  each  scientific  staff  member’s  skills  and 
ensures  coordination  among  closely  related  projects.  The 
scientific  staff  members  are  supported  by  an  exceptionally 
efficient  administrative  and  technical  staff.  All  of  the 
research  effort  is  closely  coordinated  with  ARIARDA 
personnel . 
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DEVELOPMENT  OF  A  TASK  ANALYSIS /WORKLOAD 
OPERATOR  SIMULATION  SYSTEM  (TOSS) 

Ms.  Laura  A.  Fulford,  Project  Director 


Background 

Anacapa  Sciences  researchers,  under  contract  to  the  Army 
Research  Institute  Aviation  Research  and  Development  Activity 
( ARIARDA) ,  Fort  Rucker,  Alabama,  developed  a  task  analysis/ 
workload  (TAWL)  methodology  for  predicting  operator  workload 
during  the  conceptual  phase  of  new  system  development.  The 
methodology  was  first  applied  to  the  Army's  Light  Helicopter 
Family  (LHX)  aircraft  (McCracken  &  Aldrich,  1984;  Aldrich, 
Craddock,  &  McCracken,  1984;  Aldrich,  Szabo,  &  Craddock, 

1986)  . 


Subsequently,  Anacapa  personnel  refined  the  mission/ 
task/workload  analysis  methodology  and  produced  operator 
workload  prediction  models  for  the  AH-64A  (Szabo  &  Bierbaum, 
1986),  the  UH-60  (Bierbaum,  Szabo,  &  Aldrich,  1989),  and  the 
CH-47  aircraft  (Bierbaum  &  Aldrich,  1989) .  Each  of  the 
original  workload  prediction  models  was  programmed  in  FORTRAN 
77  on  a  Perkin-Elmer  3210. 


Need 

The  FORTRAN  77  programs  for  the  LHX,  AH-64A,  and  UH-60 
workload  prediction  models  incorporate  the  model  decision 
rules  into  the  actual  program  code .  Time-consuming 
recompilations  of  the  programs  are  required  to  incorporate 
even  minor  changes  in  the  models.  A  TAWL  operator  simulation 
system  (TOSS)  is  required  that  reduces  the  development  time 
for  implementing  changes  to  existing  models  or  creating  new 
models . 


Research  Objective 

The  primary  objective  of  this  project  is  to  develop  a 
software  system  that  (a)  can  incorporate  model  changes 
without  rewriting  and  recompiling  the  software  and  (b)  is 
powerful  and  flexible  enough  to  exercise  any  of  the  workload 
prediction  models  developed  with  the  refined  TAWL  method¬ 
ology.  In  addition,  the  software  system  should  be  easy  to 
use,  portable,  and  easy  to  modify  for  the  development  of  new 
workload  prediction  models. 
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TOSS  Design 


The  TOSS  software  uses  data  files  to  store  all  model 
information  peculiar  to  a  specific  aircraft.  This  design 
approach  enables  the  programmer: 

•  to  change  an  existing  model's  execution  by  changing 
its  data  files,  thereby  eliminating  the  need  to 
rewrite  and  recompile  the  program  to  incorporate  the 
changes,  and 

•  to  implement  new  computer  models  developed  with  the 
refined  methodology  merely  by  creating  a  new  set  of 
data  files. 

Although  the  technique  of  using  data  files  to  store  all 
model  information  resolved  the  primary  problems  described 
above,  the  data  entry  of  model  information  using  an  editor  is 
time  consuming  and  subject  to  errors.  A  data  base  management 
system  with  specialized  routines  was  designed  for  entering 
and  updating  all  of  the  data  used  in  the  workload  prediction 
models.  Each  specialized  routine  features  customized  error 
checks  to  help  ensure  the  validity  of  the  data  files.  The 
most  critical  data  files  are  protected  by  automatic  backup 
procedures . 

A  simple  and  consistent  user-interface  was  developed  to 
produce  a  system  that  is  easy  to  use.  The  software  system 
was  developed  for  use  on  the  IBM  AT  compatible  computer  to 
meet  the  objective  of  portability.  Turbo  Pascal  was  selected 
as  the  development  language  to  meet  the  objective  of  easy 
modification . 


TOSS  Modifications 

During  the  current  contract  year,  numerous  modifications 
were  made  to  improve  the  basic  TOSS  software.  First,  a 
directory  utility  was  developed  to  provide  the  following 
system  capabilities: 

•  the  capability  to  build  and  work  with  a  data  base  in 
any  directory  (earlier  versions  were  limited  to  the 
root  directory  or  a  first-level  directory); 

•  the  capability  to  add,  delete,  and  rename  directories; 
and 

•  the  capability  to  rename,  copy,  and  move  files. 

Second,  the  workload  prediction  model  was  upgraded  so 
that  it  can  process  11  random  functions  per  segment  decision 
rule  rather  than  the  previous  maximum  of  8 .  In  addition,  a 
new  set  of  data  structures  was  developed  that  utilize 
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pointers  rather  than  arrays  for  the  model  execution  routine. 
The  new  structures  use  less  computer  memory  and  store  data 
more  efficiently  than  the  array  structures.  The  data  base 
management  system  was  also  upgraded  to  manage  700  tasks 
rather  than  the  previous  maximum  of  689. 

Third,  a  menu  of  output  options  was  added  to  the  model 
execution  routine.  The  menu  allows  a  user  to  execute  a  model 
and  produce  (a)  no  output,  (b)  a  simulation  listing,  (c)  an 
abbreviated  simulation  listing,  (d)  data  files  of  operator 
workload  totals,  or  (e)  a  listing  of  the  tasks  performed 
within  the  segment. 


At  the  request  of  ARIARDA,  TOSS  was  upgraded  to  a 
version  3.0.  Version  3.0  was  designed  to  accommodate  up  to 
four  crewmembers  in  one  model  rather  than  the  previous 
maximum  of  two  crewmembers.  A  more  consistent  user  interface 
was  developed  for  TOSS  3.0.  With  the  previous  data  entry 
routines,  different  responses  were  required  to  complete 
different  entries.  In  version  3.0,  a  user  must  press  the 
"Enter"  key  to  complete  a  response  in  every  situation  except 
for  menu  prompts  and  questions  that  require  a  "Y"  or  "N" 
answer.  Furthermore,  the  scales  in  the  workload  data  file 
were  converted  from  character  data  to  real  numbers  ranging 
from  0.0  to  99.99.  Finally,  a  conversion  program  (Convert3) 
was  created  to  allow  users  to  convert  their  previous  data 
files  to  the  version  3.0  format. 

In  May  1989,  TOSS  version  3.0,  Convert3,  a  draft  copy  of 
the  guide  for  using  TOSS,  and  an  archived  copy  of  the  source 
code  were  delivered  to  ARIARDA  for  distribution. 


During  the  remainder  of  the  contract  year,  further 
modifications  were  made  to  upgrade  TOSS  to  version  3.1. 
Version  3.1  makes  it  possible  to  compare  workload  between 
different  aircraft.  In  earlier  versions,  random  functions 
were  executed  at  random  times  during  the  segment,  and  the 
number  of  times  the  functions  were  executed  was  random. 
Direct  comparison  of  the  same  mission  segment  for  different 
aircraft  was  impossible  unless  the  random  functions  happened 
to  be  executed  the  same  number  of  times  in  each  model.  In 
TOSS  3.1,  an  option  in  the  system  parameters  section  allows 
the  analyst  to  set  the  number  of  times  that  random  functions 
and  tasks  are  executed. 
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Four  changes  were  made  to  the  output  options  in  TOSS 
3.1.  First,  a  user  can  send  a  detailed  listing  of  the  data 
to  a  text  file  that  can  be  either  viewed  on  the  computer 
screen  or  printed.  Second,  the  output  from  the  task, 
function,  or  segment  names  can  be  printed  numerically  or 
alphabetically . 

Third,  version  3.1  of  TOSS  allows  for  the  selection  of 
multiple  options  in  one  execution  of  the  model.  In  version 
3.0  of  TOSS,  the  model  had  to  be  performed  separately  for 
each  needed  option.  Fourth,  the  standard  output  files 
include  basic  statistics  (e.g.,  mean,  standard  deviation) 
that  are  computed  for  each  workload  component  for  each 
crewmember  during  each  segment . 

Three  modifications  were  made  to  the  data  base  manage¬ 
ment  system  in  version  3.1.  First,  the  user  is  able  to 
indicate  that  a  segment  is  to  end  after  a  specific  function 
is  performed.  Second,  version  3.1  models  the  discrete  random 
tasks  independently.  This  software  modification  allows  the 
crewmember  and  the  probability  of  occurrence  to  be  specified 
for  each  of  the  discrete  random  tasks  in  a  function  rather 
than  for  a  set  of  random  tasks.  Third,  the  maximum  number  of 
tasks  was  increased  from  700  to  800  and  the  maximum  number  of 
interrupts  was  increased  from  10  to  11. 

Finally,  changes  were  made  in  the  esthetics  of  TOSS 
version  3.1.  Version  3.1  uses  color,  shadow-box,  and 
windowing  routines  throughout  the  application.  To  maximize 
visibility  on  different  computer  monitors,  the  user  has  the 
option  of  executing  the  programs  using  either  a  color  or  a 
monochrome  monitor. 


Five  ancillary  programs  and  procedures  were  developed  to 
support  version  3.1.  First,  an  update  program  (Update31)  was 
developed  to  allow  users  of  TOSS  version  3.0  to  convert  their 
data  files  to  the  version  3.1  format. 

Second,  three  workload  graphing  programs  were  created. 
The  first  program  graphs  operator  workload  totals  that  are 
produced  during  the  execution  of  a  TOSS  model.  The  second 
program  produces  a  bar  chart  to  compare  the  average  workload 
for  each  crewmember  in  two  different  models.  The  third 
graphing  program  produces  a  bar  chart  to  compare  the  average 
workload  in  a  segment  for  two  crewmembers  in  the  same  model. 
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Finally,  a  program  that  contains  three  utility 
procedures  was  developed  for  TOSS  3.1.  One  utility  procedure 
lists  all  the  functions  that  contain  a  particular  task.  A 
similar  utility  procedure  lists  all  the  segments  that  contain 
a  particular  function.  The  third  utility  procedure  converts 
the  ordinal  workload  scales  to  interval  scales  in  the  AH-64A 
model . 


User's  Guide 

Between  February  and  October  1989,  the  draft  guide  for 
using  version  3.0  of  TOSS  was  edited  and  revised.  The  guide 
is  divided  into  two  major  sections.  Part  I  is  an  overview  of 
the  TAWL  methodology  and  describes  the  three  major  stages 
required  to  develop  a  workload  prediction  model:  a  task/ 
workload  analysis,  the  construction  of  the  model,  and  the 
execution  and  debugging  of  the  model.  This  section  provides 
managers  with  sufficient  information  to  decide  if  TAWL/TOSS 
is  an  appropriate  analytical  tool  for  analyzing  workload  on  a 
particular  system. 

Part  II  is  a  step-by-step  guide  for  using  TOSS.  The 
guide  is  structured  to  parallel  the  hierarchical  set  of 
program  command  menus.  For  example,  each  of  the  six  selec¬ 
tions  present  on  the  main  menu  of  the  program  is  represented 
as  a  section  in  Part  II. 

Parts  I  and  II  are  supplemented  by  a  glossary  and  three 
appendixes  that  provide  examples  of  workload  rating  scales, 
model  data  bases,  and  model  outputs. 


Work  Projected 

The  revised  guide  for  using  TOSS  version  3.0  will  be 
submitted  to  ARIARDA  in  October  1989.  TOSS  version  3.1  will 
be  completed,  tested,  and  distributed,  along  with  the  update 
program,  utility  program,  and  an  updated  copy  (version  3.1) 
of  the  guide. 
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THE  AH-64  WORKLOAD  PREDICTION  MODEL 
Dr.  David  B.  Hamilton,  Project  Director 


Background 

The  Army's  Air/Land  Battle  2000  scenario  presents  a 
high-threat  environment  that  will  place  heavy  workload 
demands  on  combat  helicopter  operators.  To  increase  mission 
effectiveness  in  this  environment,  the  latest  Army  heli¬ 
copters  have  been  equipped  with  advanced  technology.  This 
technology  includes  electronic  sensor  capabilities  that 
increase  both  the  amount  and  fidelity  of  information 
available  to  the  operators. 

The  AH-64A  Apache  attack  helicopter  is  equipped  with  the 
most  advanced  technology  of  any  helicopter  currently  in  the 
U.S.  Army  inventory.  It  is  the  first  Army  aircraft  equipped 
with  flight  and  weapon  systems  that  allow  missions  to  be 
conducted  at  night  and  under  adverse  weather  conditions.  The 
increased  mission  capabilities  of  the  AH-64A  aircraft  have 
dramatically  increased  the  amount  of  information  that  the 
operators  must  process.  The  AH-64A  Apache  is  equipped  with 
automated  flight  and  combat  (acquisition,  targeting,  and 
engagement)  technology  that  is  intended  to  reduce  crew  work¬ 
load.  In  some  instances,  however,  the  tasks  required  to  use 
the  technology  have  either  increased  workload  or  simply 
changed  the  nature  of  the  task  without  decreasing  workload. 

High  workload,  in  turn,  reduces  mission  effectiveness, 
increases  system  manning  requirements,  and  increases  the 
training  necessary  for  acquiring  and  maintaining  flight 
proficiency . 

One  reason  that  technology  has  failed  to  reduce  operator 
workload  in  Army  aircraft  is  that  human  factors  concepts  were 
not  adequately  considered  during  the  early  stages  of  system 
design.  For  example,  many  of  the  subsystems  in  the  AH-64A  were 
not  integrated  to  simplify  the  man-machine  interface  and  reduce 
the  operator  workload  in  the  cockpit.  In  the  past,  no  method¬ 
ology  existed  for  assessing  the  workload  demands  of  emerging 
aviation/weapon  systems  prior  to  their  development.  However, 
researchers  from  the  U.S.  Army  Research  Institute  Aviation 
Research  and  Development  Activity  ( ARIARDA)  and  Anacapa 
Sciences  have  developed  a  methodology  for  predicting  the 
workload  demands  placed  on  the  crewmembers  by  the  advanced 
technology  proposed  for  the  light  helicopter  family  (LHX) 
aircraft . 

Recently,  Anacapa  researchers  have  refined  the  LHX 
methodology  to  support  its  application  in  evaluating  operator 
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workload  in  existing  or  developmental  weapon  systems;  the 
refined  methodology  is  called  the  Task  Analysis/Workload 
(TAWL)  methodology  (see  Bierbaum,  Fulford,  &  Hamilton,  1989) . 
In  addition,  computer  support  for  the  methodology  has  been 
developed  and  named  the  TAWL  Operator  Simulation  System 
(TOSS)  . 

The  methodology  takes  a  multidimensional  view  of  human 
capabilities  that  enables  the  system  engineer  to  identify 
modifications  that  shift  operator  workload  from  one  domain  to 
another.  For  example,  technology  that  reduces  an  aviator's 
need  to  maintain  physical  control  of  system  functions  often 
increases  the  aviator’s  role  as  a  monitor.  Thus,  advanced 
technology  may  decrease  operators'  psychomotor  workload  and 
increase  their  cognitive  workload.  Because  of  the  limited 
capacity  of  human  cognitive  ability,  system  designers  must 
avoid  shifting  all  the  workload  associated  with  aircraft 
operations  into  the  cognitive  domain  (or  into  any  other 
single  domain) .  The  TAWL  methodology,  with  its  second-by- 
second  estimate  of  operator  workload,  allows  the  system 
engineer  to  utilize  all  the  operators'  capabilities  better 
and,  consequently,  increase  system  effectiveness. 

The  Army  is  currently  developing  an  improved  AH-64  heli¬ 
copter  called  the  Longbow  Apache.  The  man-machine  interface 
will  change  substantially  with  the  incorporation  of  two 
touch-screen  multifunction  displays  and  the  removal  of  the 
majority  of  the  switches  and  dials.  The  major  additions  in 
the  Longbow  Apache  will  be  the  Airborne  Target  Handover 
System  (ATHS) ,  an  Integrated  Avionics  package,  and  the 
Airborne  Adverse  Weather  Weapon  System  (AAWWS) . 

Need 

The  Aviation  Systems  Command  (AVSCOM)  requested  that 
ARIARDA  apply  the  TAWL  methodology  to  evaluate  the  operator 
workload  in  the  AH-64A  Apache  and  in  the  Longbow  Apache 
currently  being  developed.  In  response  to  AVSCOM' s  request 
for  support,  ARIARDA  tasked  Anacapa  to  conduct  the  required 
research . 


Research  Objectives 

The  overall  objective  of  the  AH-64  workload  prediction 
research  is  to  determine  the  effect  that  advanced  technology 
is  likely  to  have  on  the  workload  of  AH-64  attack  helicopter 
crewmembers.  The  research  is  divided  into  the  following 
three  specific  objectives: 
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•  determine  the  operator  workload  for  the  current 
configuration  of  the  AH-64A  aircraft, 

•  predict  the  effect  that  Longbow  design  modifications 
will  have  on  crew  workload,  and 

•  identify  the  AH-64  mission  functions  and  subsystems 
for  which  design  modifications  will  be  most  beneficial 
in  reducing  crew  workload. 

The  workload  predictions  yielded  by  the  research  will 
estimate  the  crewmember  workload  in  the  AH-64A  and  Longbow 
Apache  and  provide  a  workload  criterion  for  evaluating  the 
development  of  the  helicopter. 


Research  Approach 

The  research  for  meeting  these  objectives  is  divided 
into  three  projects: 

•  development  of  a  model  to  predict  AH-64A  crewmember 
workload, 

•  validation  of  the  AH-64A  workload  prediction  model  and 
the  TAWL  methodology,  and 

•  assessment  of  the  workload  effects  of  Longbow  Apache 
design  changes. 

Each  of  the  projects  is  described  more  fully  below,  following 
a  brief  description  of  the  work  completed  under  a  former 
contract . 


Szabo  and  Bierbaum  (1986)  conducted  a  task/workload 
analysis  of  all  phases  of  the  AH-64A  attack  mission.  A 
composite  mission  scenario  was  developed  from  five  mission 
profiles  that  assumed  optimal  flight  conditions.  In  the 
composite  scenario,  the  pilot's  primary  function  is  to  fly 
the  aircraft,  and  the  gunner’s  primary  function  is  to  acquire 
and  engage  targets.  No  reconnaissance  or  team  leader  func¬ 
tions  are  performed  by  the  crew.  Seven  mission  phases  were 
identified  and  divided  into  52  unique  mission  segments.  The 
segments  were  further  divided  into  159  unique  functions  with 
688  individual  tasks  necessary  to  the  mission.  Finally,  the 
subsystem,  crewmember,  and  duration  for  each  task  was 
identified . 
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Development  of  the  AH- 64  Workload  Prediction -Modal 

The  development  of  a  computer  model  to  predict  workload 
for  the  AH-64A  Apache  crewmembers  will  be  divided  into  four 
tasks.  The  initial  task  will  be  to  enter  the  mission/task/ 
workload  analysis  data  into  a  computer  data  base.  The  second 
task  will  be  to  develop  and  enter  the  function  and  segment 
decision  rules  into  the  computer.  A  function  summary  sheet 
will  be  developed  for  each  unique  function  to  identify  the 
specific  tasks  performed  by  each  crewmember.  Function 
decision  rules  will  be  written  using  the  function  summary 
sheets.  Function  decision  rules  specify  the  sequence  and 
time  for  performing  the  tasks.  Following  the  development  of 
the  function  decision  rules,  segment  summary  sheets  and 
decision  rules  will  be  developed.  The  segment  decision  rules 
specify  the  sequence  and  time  for  performing  the  functions  in 
each  mission  segment. 

In  the  third  task,  the  TOSS  software  will  be  utilized  to 
automate  the  analysis  of  workload.  The  computer  program  will 
use  the  function  and  segment  decision  rules  to  combine  the 
tasks  to  form  functions  which,  in  turn,  will  be  combined  to 
form  segments.  The  computer  program  will  simulate  the 
sequence  of  tasks  that  each  crewmember  must  perform  to  accom¬ 
plish  the  mission.  From  this  simulation,  the  program  will 
generate  total  workload  estimates  for  each  of  the  five 
workload  components  (visual,  auditory,  cognitive,  psycho¬ 
motor,  and  kinesthetic)  by  summing  the  individual  workload 
ratings  for  all  the  tasks  that  are  performed  concurrently. 

The  total  component  workload  predictions  will  be  generated 
for  every  half-second  interval  in  the  segment.  The  estimates 
of  component  workload  will  identify  points  on  the  mission 
timeline  where  excessive  workload  (i.e.,  overload)  will 
occur.  Thus,  predictions  of  total  workload  associated  with 
the  performance  of  concurrent  and  sequential  tasks  in  the 
AH-64A  baseline  configuration  will  be  generated. 

In  the  fourth  task,  the  results  of  the  simulation  will 
be  reviewed  to  identify  and  correct  any  errors  in  the  task/ 
workload  analysis  data  base.  In  addition,  AH-64A  subject 
matter  experts  will  review  the  computer  simulation  of  the 
crewmembers'  actions  during  each  mission  segment  to  ensure 
that  the  model  conforms  with  typical  crewmember  actions. 


Validation  of  the  AH-64A  Workload  Prediction  Model 


During  the  second  project,  the  TAWL  methodology  and  the 
AH-64A  workload  prediction  model  will  be  validated.  The 
workload  predictions  yielded  by  the  model  will  be  evaluated 


14 


by  conducting  part-mission  and  full-mission  simulation 
research.  In  each  instance,  predictions  of  workload  for 
specific  tasks  will  be  compared  with  objective  measures  of 
primary  task  performance,  physiological  measures  of  workload, 
and  subjective  measures  of  workload  (e.g.,  Reid, 

Shingledecker ,  &  Eggemeier,  1981;  Hart  &  Staveland,  1987). 

Finally,  the  results  of  the  validation  research  will  be 
used  to  refine  the  model.  The  research  to  validate  the 
AH- 64 A  model  will  not  only  establish  the  accuracy  of  the 
predictions  of  AH-64A  workload  prediction  model,  but  will 
also  establish  the  utility  of  the  TAWL  methodology  for 
producing  valid  models  of  operator  workload. 


Workload  Analysis  of  the  Longbow .Apache 

During  this  project,  the  AH-64A  computer  model  will  be 
modified  and  exercised  to  predict  how  crew  workload  might  be 
affected  by  the  changes  made  to  the  Longbow  model.  The 
project  consists  of  the  following  steps: 

•  establish  a  secure  computer  system  for  developing  the 
model, 

•  identify  the  design  changes  that  affect  the  operation 
of  the  system, 

•  conduct  a  task/workload  analysis  for  each  change, 

•  develop  the  function  and  segment  decision  rules  for 
the  changes, 

•  exercise  the  Longbow  model  to  yield  estimates  of 
workload,  and 

•  compare  the  estimates  of  workload  for  the  baseline  and 
Longbow  configurations. 

The  results  of  this  project  will  be  used  to  estimate  the 
differences  in  crewmember  workload  between  the  AH-64A  and  the 
Longbow  Apache.  The  estimates,  in  turn,  will  assist  design 
engineers  in  identifying  the  configuration  of  the  AH-64  that 
produces  lower  operator  workload. 


Work  Completed 

Development  of  the  AH-64  Workload  Prediction  Model 

During  the  first  year  of  the  current  contract,  both  the 
task /workload  analysis  and  the  decision  rules  were  exten¬ 
sively  reviewed  and  revised.  In  addition,  a  preliminary 
version  of  the  computer  model,  using  a  Perkin-Elmer  mini¬ 
computer  and  FORTRAN  language,  was  developed.  The  model  was 


exercised  to  produce  preliminary  analyses  of  workload  for 
each  of  the  mission  segments. 

During  the  second  contract  year,  the  AH-64A  mission/ 
task/workload  analysis  was  reprogrammed  using  the  TOSS 
software.  The  AH-64A  workload  model  was  exercised  to  produce 
mission  segment  printouts,  which  were  reviewed  to  ensure  that 
the  computer  model  accurately  simulated  the  function  and 
segment  decision  rules.  In  some  cases,  the  function  and 
segment  decision  rules  were  revised. 

During  the  current  contract  year,  Anacapa  researchers 
developed  the  capability  to  graph  the  output  of  TOSS  models. 

A  review  of  the  graphic  output  identified  a  number  of  inaccu¬ 
racies  in  the  decision  rules  that  require  revision.  By  the 
end  of  the  current  contract  year,  the  AH-64A  baseline  work¬ 
load  prediction  model  was  being  revised  to  increase  its 
accuracy.  The  changes  included  substituting  interval  rating 
scales  for  ordinal  workload  scales,  revising  the  task/ 
workload  analysis,  and  revising  the  function  and  segment 
decision  rules  . 
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During  the  current  contract  year,  a  draft  validation 
research  plan  was  written.  The  research  plan  stipulates  the 
measurement  of  several  objective  and  subjective  measures  of 
operator  workload  that  will  be  used  as  validation  criteria. 
In  addition,  a  request  was  prepared  and  submitted  to  Forces 
Command  (FORSCOM)  for  support  in  validating  the  AH-64A  model 


Workload  Analysis  of  the  Longbow  Apache 

During  the  current  contract  year,  a  secure  computer 
system  was  established  for  developing  the  Longbow  Apache 
workload  prediction  model.  The  major  modifications  to  the 
Longbow  Apache  were  identified,  changes  to  the  composite 
mission  scenario  resulting  from  the  modifications  were 
analyzed,  and  the  additional  segments  needed  to  model  the 
changes  were  identified.  At  the  close  of  the  current 
contract  year,  work  had  begun  on  the  task/workload  analysis 
of  the  Longbow  Apache . 
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Work  Projected 


Development  of  the  AH-64  Workload  Prediction  Model 

During  the  next  contract  year,  the  revised  model  will  be 
exercised  to  produce  estimates  for  each  of  the  52  unique 
mission  segments.  These  predictions  will  be  reviewed  to 
produce  the  final  baseline  version  of  the  AH-64A  model.  The 
technical  report  by  Szabo  and  Bierbaum  (1986)  will  be  revised 
to  reflect  the  final  version  of  the  AH-64A  model.  The 
workload  predictions  produced  by  the  exercise  of  the  model 
will  be  described  in  a  research  report. 


Validation  of  the  AH-64A  Workload  Prediction  Model 

The  draft  research  plan  for  validating  the  AH-64A  model 
will  be  completed  and  submitted  to  ARIARDA  for  approval. 

Upon  completion  of  the  AH-64A  model  revisions,  ARIARDA 
approval  of  the  research  plan,  and  approval  for  using  FORSCOM 
resources,  the  validation  research  will  begin.  A  report 
describing  the  results  of  the  research  will  be  prepared  when 
the  validation  effort  is  completed. 


Workload  Analysis.  o.f  the  ..Longbow. .Apache 

The  task/workload  analysis  will  continue.  The  necessary 
function  and  segment  decision  rules  will  be  developed  and  the 
model  will  be  programmed  using  the  TOSS  software.  The  model 
will  be  exercised  and  the  workload  predictions  will  be  com¬ 
pared  to  AH-64A  model  predictions.  Finally,  a  research 
report  will  be  prepared  to  document  the  results  of  the 
model's  predictions  and  the  comparisons  between  the  two  AH-64 
models . 
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UH/MH-60  AND  CH/MH-47  TASK/WORKLOAD  ANALYSES 
Mr.  Carl  R.  Bierbaum,  Project  Director 


Background 

The  Special  Operations  Aircraft  (SOA)  Program  Manager's 
(PM)  office  at  the  Army's  Aviation  Systems  Command  has  been 
tasked  to  develop  an  MH-60K  and  an  MH-47E  aircraft  to  support 
the  Special  Operations  Forces.  The  SOA  will  consist  of 
existing  CH-47D  and  UH-60A  airframes  with  increased  power  and 
new  integrated  cockpits.  The  integrated  cockpit  will  replace 
the  existing  instrument  and  gauge  configuration  in  both  the 
CH-47D  and  UH-60A  aircraft  with  four  multifunction  display 
(MFD)  units. 

The  effect  that  the  high  technology  modifications  being 
proposed  for  the  MH-60K  and  the  MH-47E  may  have  on  crewmember 
workload  must  be  evaluated  to  ensure  that  the  crewmembers  can 
utilize  the  many  MFD  options  effectively.  Anacapa  Sciences 
personnel,  under  contract  to  the  Army  Research  Institute 
Aviation  Research  and  Development  Activity  (ARIARDA) ,  have 
developed  a  methodology  for  predicting  operator  workload 
during  system  design.  The  workload  prediction  methodology 
was  developed  during  the  design  of  the  Army's  light  heli¬ 
copter  aircraft  (LHX)  (Aldrich,  Craddock,  &  McCracken,  1984; 
Aldrich,  Szabo,  &  Craddock,  1986) .  The  LHX  methodology  was 
subsequently  refined  and  used  to  develop  baseline  workload 
prediction  models  for  the  AH-64A  Apache  (Szabo  &  Bierbaum, 
1986) ,  the  UH-60A  Black  Hawk  (Bierbaum,  Szabo,  &  Aldrich, 

1989) ,  and  the  CH-47D  Chinook  (Bierbaum  &  Aldrich,  1989) . 
Because  ARIARDA  had  developed  a  successful  methodology  for 
conducting  mission/task  analyses  and  predicting  workload,  the 
SOA  Aviation  Project  Office  requested  that  ARIARDA  develop  a 
SOA  scenario,  conduct  the  mission/task  analyses,  and  predict 
the  crewmember  workload  for  the  MH-60K  and  MH-47E  aircraft. 


Research  Objectives 

The  overall  objective  of  this  research  area  is  to 
determine  the  effect  that  the  integrated  cockpit 
modifications  are  likely  to  have  on  the  workload  of  UH-60A 
and  CH-47D  crewmembers.  Specifically,  the  research  is 
designed  to: 

•  provide  a  mission/task  analysis  of  the  UH-60A  and  the 
MH-60K, 

•  determine  the  effect  that  the  proposed  MH-60K  modi¬ 
fications  will  have  on  crew  workload, 
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•  provide  a  mission/task  analysis  of  the  CH-47D  and  the 
MH-47E,  and 

•  determine  the  effect  that  the  proposed  MH-47E  modi¬ 
fications  will  have  on  crew  workload. 


Research  Approach 

The  approach  selected  for  meeting  the  research  objec¬ 
tives  is  a  refinement  of  the  Task  Analysis/Workload  (TAWL) 
methodology  that  was  developed  for  the  LHX  and  the  AH-64A 
research  projects.  The  UH/MH-60  and  CH/MH-47  workload 
analyses  are  divided  into  the  following  four  projects: 

•  In  the  first  project,  a  baseline  mission/task  analysis 
of  crew  workload  will  be  conducted  and  a  computer 
model  of  crewmember  workload  will  be  developed  for  the 
UH-60A  helicopter. 

•  In  the  second  project,  a  baseline  mission/task 
analysis  of  crew  workload  will  be  conducted  and  a 
computer  model  of  crewmember  workload  will  be 
developed  for  the  CH-47D  helicopter. 

•  In  the  third  project,  a  mission/task  analysis  of  crew 
workload  will  be  conducted  and  a  computer  model  of 
crewmember  workload  will  be  developed  for  the  MH-60K 
helicopter.  The  UH-60A  and  the  MH-60K  workload 
prediction  models  will  be  exercised  and  compared  to 
predict  the  effect  that  the  MH-60K  design  modifica¬ 
tions  are  likely  to  have  on  crewmember  workload. 

•  In  the  fourth  project,  a  mission/task  analysis  of  crew 
workload  will  be  conducted  and  a  computer  model  of 
crewmember  workload  will  be  developed  for  the  MH-47E 
helicopter.  The  CH-47D  and  the  MH-47E  workload 
prediction  models  will  be  exercised  and  compared  to 
predict  the  effect  that  the  MH-47E  design  modifica¬ 
tions  are  likely  to  have  on  crew  workload. 


Work  Completed 

UH-60A  Task  Analysis  and  Workload  Prediction  Model 

The  development  of  the  UH-60A  workload  prediction  model 
was  completed  during  the  first  contract  year  and  was  reported 
in  the  1987  Annual  Summary  Report  (Bierbaum,  1988)  and  in  a 
research  report  by  Bierbaum,  Szabo,  and  Aldrich  (1989) . 
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CH-47D  Task  Analysis  and  Workload  Prediction  Model 

The  development  of  the  CH-47D  workload  prediction  model 
was  completed  during  the  second  contract  year  and  was 
reported  in  the  1988  Annual  Summary  Report  (Bierbaum,  1989) . 
A  report  of  the  CH-47D  workload  prediction  model  was 
completed  in  January  1989  and  submitted  to  ARIARDA  (Bierbaum 
&  Aldrich,  1989) . 


MH-60K  Task  Analysis  and  Workload .. Pr.edi.ct.iQD_MQ.dsl 

The  development  of  a  mission  scenario  and  the  mission 
phases  was  completed  during  the  second  contract  year  and  was 
reported  in  the  1988  Annual  Summary  Report  (Bierbaum,  1989) . 
During  the  current  contract  year,  work  continued  on  the 
MH-60K  task  analysis  and  on  the  development  of  an  MH-60K 
workload  prediction  model.  The  completed  project  tasks  are 
briefly  described  below. 

Develop  a  mission  scenario.  The  MH-60K  mission  begins 
with  a  departure  from  a  base  site.  The  pilot  flies  contour 
from  the  base  to  a  rendezvous  point,  where  air-to-air 
refueling  is  accomplished.  After  refueling,  the  pilot  flies 
nap-of-the-earth  (NOE)  from  the  rendezvous  point  to  the 
landing  zone  (LZ) .  The  pilot  then  flies  back  to  a  rendezvous 
point  for  refueling  and  continues  to  the  base.  The  complete 
mission  is  conducted  at  night  with  night  vision  goggles. 
Preflight  and  postflight  activities  are  not  included  in  the 
analysis . 

Divide  the  mission  scenario.  The  mission  scenario  was 
divided  into  five  phases  for  analysis.  The  five  mission 
phases  were  further  divided  into  15  unique  segments.  Some 
segments  are  used  in  more  than  one  of  the  phases.  The  number 
of  segments  in  each  phase  is  listed  below: 

•  Departure  (Base)  -  3, 

•  Enroute  (Base-Rendezvous)  -  4, 

•  Enroute  (Rendezvous-LZ )  -  4, 

•  Enroute  (LZ-Rendezvous )  -  4,  and 

•  Enroute  (Rendezvous-Base)  -  3. 

Divide  the  segments  into  functions  and  taste.  The  15 
unique  segments  were  divided  into  71  unique  functions.  Each 
function  was  described,  alphabetized,  and  assigned  a 
numerical  identification  code  for  entry  into  the  MH-60K 
workload  model.  The  71  functions  were  then  divided  into  234 
tasks.  Each  unique  task  was  described,  alphabetized  by 
object,  and  assigned  a  numerical  identification  code  for  use 
in  the  workload  model. 
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Identify  the  subsystems  associated  with  each  task. 
Seventeen  aircraft  subsystems  were  identified  on  the  MH-60K. 
The  subsystems  were  divided  into  five  major  categories: 
Engine,  Flight  Control,  Navigation,  Utility,  and  Visual.  The 
subsystems  were  coded  and  entered  into  the  workload  model  so 
that  crewmember  overloads  could  be  associated  with  the 
specific  subsystems  in  use  when  each  overload  occurred. 

Estimate  the  workload  and  time  required  £or_e.a.ch  task. 

A  short  verbal  description  of  each  of  the  five  workload 
components  (visual,  auditory,  cognitive,  psychomotor,  and 
kinesthetic)  was  written  for  each  task.  The  descriptors  were 
then  compared  to  the  verbal  anchors  contained  in  7-point 
rating  scales  designed  for  rating  each  of  the  workload 
components.  The  appropriate  scale  value  for  each  component 
was  entered  into  the  MH-60K  model  as  an  estimate  of  the  task 
workload . 

The  Integrated  Avionics  Subsystem  (IAS)  to  be  installed 
in  the  MH-60K  cockpit  was  not  available  for  an  empirical 
analysis  of  the  time  needed  to  perform  each  task.  Instead, 
the  time  assigned  to  each  task  was  estimated  by  subject 
matter  experts  in  the  IAS  Crewstation  Working  Group. 

Develop  and  exercise  the  MH-60K  model.  The  specific 
tasks  identified  during  the  analysis  were  entered  into  the 
TAWL  Operator  Simulation  System  (TOSS)  data  files.  The 
researchers  developed  the  decision  rules  required  to  combine 
specific  tasks  into  functions  and  to  combine  the  functions 
into  segments.  The  decision  rules  were  then  programmed  and 
the  workload  prediction  model  was  exercised  to  provide 
estimates  of  component  workload  at  each  half-second  interval 
for  each  segment  in  the  scenario. 


MH-47E  Task  Analysis  and  Workload  Prediction  Model 

During  the  current  contract  year,  work  continued  on  the 
MH-47E  task  analysis  and  on  the  development  of  an  MH-47E 
workload  prediction  model.  The  mission  scenario,  mission 
phases,  segments,  and  functions  for  the  MH-47E  are  the  same 
as  those  identified  for  the  MH-60K.  However,  only  230  tasks 
were  identified  for  the  MH-47E. 

Develop  and  exercise  the  MH-47E  model.  The  specific 
tasks  identified  during  the  analysis  were  entered  into  data 
files  for  the  workload  prediction  model.  The  researchers 
developed  decision  rules  to  combine  the  tasks  into  functions 
and  to  combine  the  functions  into  segments.  The  decision 
rules  were  then  programmed  and  the  workload  prediction  model 
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was  exercised  to  provide  estimates  of  component  workload  at 
each  half-second  interval  for  each  segment  in  the  scenario. 


Work  Projected 

MH- 60K  and  MH-47E  Workload  Prediction  Models 

During  the  next  contract  year,  the  MH-60K  workload 
predictions  will  be  compared  with  the  UH-60A  workload  and  the 
MH-47E  workload  predictions  will  be  compared  with  the  CH-47D 
workload.  Finally,  two  research  reports  will  be  prepared  to 
document  the  results  of  the  analyses. 
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VALIDATION  OF  THE  LHX  WORKLOAD  PREDICTION  MODEL 
Mr.  Theodore  B.  Aldrich,  Project  Director 


Background 

Models  that  predict  operator  workload  can  be  useful 
tools  for  human  factors  engineers  who  are  addressing  human 
capabilities  and  limitations  during  the  design  of  advanced 
technology  weapon  systems.  Accordingly,  Anacapa  Sciences 
researchers,  under  contract  to  the  U.S.  Army  Research 
Institute  Aviation  Research  and  Development  Activity, 
developed  a  workload  prediction  methodology  and  produced  one- 
and  two-crewmember  models  for  predicting  aviator  workload  in 
advance  of  aircraft  system  design.  The  workload  prediction 
methodology  operationally  defines  workload  in  terms  of 
attentional  demand  and  predicts  workload  associated  with 
task-level  performance.  The  Anacapa  researchers  applied  the 
workload  prediction  models  during  the  conceptual  design  phase 
of  a  proposed  multipurpose,  lightweight  helicopter  designated 
the  LHX  (Aldrich,  Szabo,  &  Craddock,  1986) . 


Need 

Neither  the  workload  predictors  used  to  develop  the 
models  nor  the  workload  predictions  yielded  by  the  models 
have  been  validated.  Workload  model  predictors  that  require 
validation  include  the: 

•  workload  ratings  assigned  to  each  task, 

•  total  workload  estimates  for  concurrent  tasks, 

•  estimated  time  required  to  perform  each  task, 

•  temporal  relationships  among  tasks,  and 

•  sequential  relationships  among  tasks. 

Specific  predictions  yielded  by  the  models  that  require 
validation  include  the  four  indexes  of  excessive  workload 
(Aldrich,  Craddock,  &  McCracken,  1984)  listed  below: 

•  component  overloads, 

•  overload  conditions, 

•  overload  density,  and 

•  subsystem  overloads. 


Research  Objectives 

This  project  is  divided  into  three  phases.  The  objec¬ 
tives  of  Phase  1  are  to  evaluate  the  reliability  of  (a)  the 
scales  used  to  rate  the  workload  components  of  each  operator 
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task  identified  during  the  LHX  workload  analyses  and  (b)  the 
workload  predictors  used  in  developing  the  LHX  workload 
prediction  model.  The  objective  of  Phase  2  is  to  obtain 
validation  data  through  part -mission  and  full-mission  flight 
simulation  research.  The  objective  of  Phase  3  is  to  refine 
the  workload  prediction  model  on  the  basis  of  the  validation 
research  results. 


Research  Approach 

The  research  plan  (Aldrich  &  Szabo,  1986)  describes  18 
tasks  that  are  required  to  accomplish  the  three  phases  of  the 
validation  research.  A  summary  of  the  research  methodology 
for  each  of  the  three  phases  is  described  below. 


EL.a.5£— 1 

In  Phase  1,  two  surveys  will  be  administered  to 
approximately  70  human  factors  scientists  who  are  familiar 
with  workload  research.  In  the  first  survey,  all  possible 
pairs  of  the  verbal  anchors  from  each  of  the  workload 
component  rating  scales  will  be  presented  to  the  workload 
subject  matter  experts  (SMEs) .  The  SMEs  will  select  the 
anchor  in  each  pair  that  requires  the  greatest  attentional 
demand.  The  results  of  this  survey  will  be  used  (a)  to 
assess  the  interrater  reliability  of  the  scale  anchors  and 
(b)  to  derive  equal  interval  scales  (eg.,  Engen,  1971)  to 
replace  the  ordinal  scales  used  in  the  original  workload 
analysis . 

The  second  survey  will  ask  the  same  SMEs  to  use  the 
workload  component  scales  to  rate  the  short  descriptors  of 
visual,  auditory,  cognitive,  and  psychomotor  components  of 
workload  for  each  task  in  the  model.  Correlational  tech¬ 
niques  will  be  used  to  evaluate  the  interrater  reliability  of 
the  workload  ratings. 


Plias.£.-2 

In  Phase  2,  part -mission  and  full-mission  simulation 
experiments  will  be  conducted  to  validate  the  workload 
estimates.  For  the  part-mission  simulation,  mini-scenarios 
will  be  generated  by  selecting  concurrent  and  sequential 
tasks  from  the  mission/task  analysis.  For  the  full-mission 
simulation,  a  composite  mission  scenario  will  be  developed  by 
selecting  segments  from  the  mission/task  analysis. 
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The  part-mission  simulation  experiments  will  be 
conducted  using  a  repeated  measures  experimental  design  in 
which  each  subject  will  fly  the  mini-scenarios  multiple 
times.  The  results  will  be  analyzed  by  correlating  the 
workload  predictors  and  the  measures  of  the  operators' 
performance  on  the  concurrent  and  sequential  tasks.  The 
correlation  coefficients  will  indicate  how  accurately  the 
workload  predictors  forecast  excessive  workload  at  the  task 
level.  To  validate  the  time  estimates  used  in  the  model,  the 
time  required  to  perform  the  various  tasks  in  the  mini¬ 
scenarios  will  be  compared  with  the  time  estimated  during  the 
task  analysis.  The  procedural  relationships  among  the  tasks 
will  be  evaluated  by  assessing  the  subjects'  ability  to 
progress  through  the  mini-scenarios  following  the  sequence  of 
tasks  specified  by  the  model. 

During  the  full-mission  simulation  experiments,  each 
trial  will  start  at  the  beginning  of  a  composite  scenario  and 
continue  without  interruption  to  the  end.  All  the  part- 
mission  simulation  data  analyses  will  be  conducted  on  the 
full-mission  simulation  data.  In  addition,  an  analysis  will 
be  performed  to  assess  the  effects  of  inserting  secondary 
tasks  into  the  composite  mission  scenario. 

The  final  task  in  Phase  2  will  be  to  compare  the  results 
from  the  part-mission  simulation  research  with  results  from 
the  full-mission  simulation  research.  The  findings  from  this 
comparison  will  be  used  to  determine  if  excessive  workload 
results  from  the  cumulative  effects  of  high  workload  over  the 
longer  times  in  the  composite  mission  scenario. 


Phase  3 


In  Phase  3,  refinements  will  be  made  to  the  workload 
prediction  model  on  the  basis  of  the  results  from  the  first 
two  phases.  First,  the  workload  component  rating  scales  will 
be  converted  from  ordinal  to  interval  scales.  Second, 
refinements  will  be  made  to  the  workload  model  algorithms  to 
reflect  the  empirical  results  of  the  part-mission  and  full- 
mission  simulation  experiments. 


Phase  .1 


Work  Completed 


The  first  survey  was  conducted  during  the  second 
contract  year.  A  survey  form  was  developed  that  presented 
all  possible  pairs  of  the  verbal  anchors  from  each  of  the 
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four  workload  component  rating  scales.  The  survey  forms  were 
mailed  to  71  SMEs  who  were  asked  to  select  the  verbal  anchor 
in  each  pair  that  was  judged  to  require  the  greatest  atten- 
tional  demand.  Completed  data  forms  were  received  from  38 
SMEs.  Kendall's  Coefficient  of  Concordance  (Siegal,  1956) 
was  used  to  assess  the  degree  of  agreement  among  the  SMEs. 

The  coefficients  for  the  four  scales  ranged  from  .39  (visual) 
to  .69  (cognitive).  All  of  the  coefficients  are  significant 
at  the  .001  level  and  indicate  moderate  agreement  among  the 
SMEs  about  the  attentional  demand  for  each  verbal  anchor. 


Zkasa— 2. 

In  October  1987,  the  new  Crew  Station  Research  and 
Development  Facility  (CSRDF)  located  at  the  the  Army's 
Aerof lightdynamics  Laboratory,  NASA  Ames,  Moffett  Field, 
California,  was  selected  as  the  most  appropriate  site  for 
conducting  the  validation  research.  The  LHX  part-mission 
simulation  research  was  scheduled  for  October  1988,  January 
1989,  and  April  1989.  The  LHX  full-mission  simulation 
research  was  scheduled  for  July  through  September  of  1989. 

In  April  1988,  CSRDF  planners  postponed  the  simulation 
research  for  October  1988  until  October  1989  because  of 
delays  in  acquiring  a  new  Compuscene  IV  visual  system  and 
because  of  changes  in  research  priorities.  CSRDF  simulation 
planners  also  stated  that  the  initial  LHX  validation  research 
should  be  full-mission  simulation  with  two  crewmembers.  This 
differs  from  the  project  research  plan  that  begins  with  the 
part-mission,  one-crewmember  simulation. 


Phase  3 

During  the  second  contract  year,  the  data  from  the  pair 
comparison  survey  were  used  to  derive  interval  values  for  the 
workload  component  scales.  The  workload  prediction  model  was 
refined  by  replacing  the  original  ordinal  values  for  each 
task  with  the  interval  values.  The  one-crewmember  workload 
model  was  exercised  with  the  new  interval  scale  values  to 
produce  estimates  of  workload  for  each  of  the  29  mission 
segments  in  the  model.  A  comparison  of  the  workload 
predicted  by  the  two  sets  of  scales  showed  that,  although 
there  are  some  differences,  the  interval  scale  estimates 
closely  correspond  to  the  ordinal  scale  estimates. 
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Work  Projected 


Because  of  the  repeated  delays  in  access  to  the  CSRDF 
simulation  facilities  and  CSRDF  changes  to  the  simulation 
design,  this  project  was  discontinued  in  April  1989.  No 
further  work  is  projected.  The  need  to  validate  the  Anacapa 
Sciences/ARIARDA  workload  prediction  methodology  will  be  met 
by  evaluating  the  AH-64  model  predictions. 
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DESIGN  AND  APPLICATION  OF  FLIGHT  SYMBOLOGY 
Dr.  Richard  Weeter,  Project  Director 


Background 

The  AH-64A  attack  helicopter  is  the  first  Army  aircraft 
to  feature  the  Pilot  Night  Vision  System  (PNVS) .  The  PNVS  is 
a  display  system  that  enables  crew  members  to  conduct  attack 
missions  at  night  and  in  adverse  weather  by  providing  an 
infrared  image  of  the  external  visual  scene.  The  PNVS 
presents  a  30°  x  40°  field  of  view  to  the  pilot's  right  eye 
via  a  1  inch  in  diameter  cathode  ray  tube  mounted  on  the 
pilot's  helmet.  A  set  of  27  symbols,  intended  to  provide 
critical  flight  and  targeting  information,  can  be  projected 
onto  the  field  of  view. 

The  PNVS  symbology  consists  of  alphanumeric,  position, 
size,  and  shape-ceded  symbols.  Some  of  the  symbols,  such  as 
the  heading  scale  and  lubber  line,  are  adaptations  of  tradi¬ 
tional  electro-mechanical  instruments  that  appear  at  fixed 
locations  on  the  display.  Many  of  the  other  symbols  are 
unique,  dynamic  representations  of  spatial  information. 
Symbols,  such  as  the  projected  center  line  of  the  aircraft  or 
the  computed  impact  points  of  weapons,  may  appear,  disappear, 
or  move  on  and  off  the  display  as  a  result  of  changes  in  the 
aircraft  or  sensor  orientation. 

Prior  to  the  development  of  the  PNVS  in  the  late  1970s, 
Schmit  (1977)  found  little  empirical  research  to  suggest  a 
basis  for  the  evaluation  of  potential  symbology  formats. 
During  the  development  of  the  PNVS  symbology  format,  Buckler 
(1978a)  described  the  state  of  empirical  research  comparing 
different  formats  as  sorely  lacking.  Furthermore,  Buckler 
(1978b)  reported  that  reconf igurable  simulators  were  not 
readily  available  to  test  alternative  symbology  formats  for 
the  PNVS. 

To  date,  no  empirical  research  has  been  identified  that 
evaluates  whether  the  PNVS  symbology  format  enhances  or 
degrades  information  transfer  during  mission  tasks.  Never¬ 
theless,  the  current  Department  of  Defense  military  standard 
for  symbology  formats,  MIL-STD-12 95A (AV) ,  is  patterned  after 
the  AH-64A  PNVS  symbology  set  (Department  of  Defense,  1984). 
In  the  foreword  of  that  document,  the  authors  acknowledge  the 
need  for  research  on  symbology  format  design.  Historically, 
however,  the  development  of  symbology  has  been  evolutionary 
rather  than  systematic  (Shrager,  1977) .  An  example  is  the 
symbology  developed  for  the  Army's  MH-60K  and  MH-47E  special 
operations  helicopters  (International  Business  Machines, 
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1988) .  Different  symbols  are  used  to  present  some  of  the 
same  basic  flight  information  represented  in  the  PNVS 
symbology  format,  but  no  information  is  publicly  available  to 
explain  how  the  new  symbols  were  developed  or  how  the  new 
symbology  format  will  affect  crew  performance. 


Need 

Currently,  there  is  no  widely  accepted  research  method¬ 
ology  for  addressing  critical  symbology  design  issues  or  for 
evaluating  the  effectiveness  of  existing  symbology  sets.  As 
a  result,  there  are  no  empirically  valid  design  criteria  for 
new  aircraft  display  symbology.  Therefore,  a  methodology  is 
needed  to  evaluate  the  effectiveness  of  the  AH-64A  PNVS 
symbology  set,  which  represents  the  current  military  standard 
in  symbology  formats. 

The  research  methodology  must  address  whether  the  symbol 
coding  dimensions  are  compatible  with  the  cognitive  processes 
of  AH-64A  crewmembers.  Ideally,  successive  experiments  will 
culminate  in  the  development  of  symbol  and  display  format 
design  criteria.  The  resultant  symbology  should  (a)  be 
compatible  with  the  known  visual  and  cognitive  capabilities 
of  aviators,  (b)  present  information  that  can  be  interpreted 
accurately  and  efficiently  under  stressful  conditions,  and 
(c)  complement  rather  than  interfere  with  information 
available  from  the  natural  external  visual  scene  and  from 
sensor-provided  imagery. 

The  Army  Aviation  Systems  Command  (AVSCOM)  tasked  the 
Army  Research  Institute  Aviation  Research  and  Development 
Activity  (ARIARDA)  to  initiate  research  to  meet  these  needs. 
Anacapa  Sciences  personnel  began  work  on  the  project  in 
February,  1987 . 


Project  Objectives 

There  are  three  objectives  of  this  project  are  (a)  to 
develop  a  methodology  for  evaluating  aircraft  display  symbol¬ 
ogy,  (b)  to  conduct  empirical  evaluations  of  the  existing 
AH-64A  PNVS  symbology,  and  (c)  to  identify  potential  design 
criteria  for  modifying  the  current  AH-64A  PNVS  symbology  and 
for  developing  future  aircraft  display  symbology. 
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Research  approach 


Following  a  review  of  the  literature,  a  selective  visual 
attention  approach  was  chosen  to  evaluate  the  PNVS  symbology 
because  it  provides  a  method  of  empirically  comparing  the 
demand  of  attending  to  visual  stimuli.  In  some  types  of 
selective  visual  attention  paradigms,  subjects  perform  funda¬ 
mental  visual  tasks  similar  to  those  required  of  pilots  using 
aircraft  visual  displays  (e.g.,  Lyon,  1987;  Williams,  1982). 
Such  experiments  have  ’-evealed  that  a  number  of  factors 
affect  attentional  performance  on  visual  tasks.  For  example, 
Eriksen  and  Hoffman  (1972)  demonstrated  that  efficient 
encoding  of  information  from  visual  displays  can  be  detri¬ 
mentally  affected  by  the  number,  nature,  and  proximity  of 
noise  elements.  Pilots  using  aircraft  visual  displays  with 
several  symbols  in  close  proximity,  a  condition  described  as 
display  clutter,  have  reported  similar  encoding  difficulties. 
Lyon  suggested  that  rapid  attention  shifts  may  be  a  measur¬ 
able  component  of  skilled  performance  in  vision  dependent 
tasks . 


The  cueing  procedure  for  the  cued  line-of-sight  (LOS) 
symbol  in  the  AH-64A  PNVS  symbology  set  was  selected  for  the 
initial  evaluation.  The  purpose  of  the  cued  LOS  symbol  is  to 
indicate  to  the  pilot  where  the  copilot-gunner  is  looking. 

The  procedure  uses  two  different  cues,  a  one-dot  cue  and  a 
two-dot  cue,  to  indicate  one  of  the  eight  search  areas  in  the 
PNVS  field  of  view.  The  cueing  dots  also  have  a  secondary 
purpose:  they  flash  to  indicate  that  an  Integrated  Helmet 

and  Display  Sighting  System  (IHADSS)  boresight  is  required. 

Three  experiments  were  designed  to  evaluate  the  cued  LOS 
symbol;  the  experiments  address  differences  in  the  one-  and 
two-dot  conditions,  differences  in  presentation  duration,  the 
effects  of  the  secondary  cueing  meaning,  and  the  effects  of 
practice  on  cueing  accuracy.  In  addition  to  evaluating  the 
cued  LOS  symbol,  the  experiments  were  designed  as  a  test  of 
the  selective  attention  paradigm  for  evaluating  symbology 
design.  If  the  results  of  the  initial  test  are  positive, 
then  additional  symbols  will  be  selected  for  evaluation. 


Work  Completed 

Between  November  1988  and  March  1989,  three  selective 
visual  attention  experiments  were  conducted  to  evaluate  the 
PNVS  cueing  procedure.  Experiment  1  was  conducted  to  eval¬ 
uate  the  task  demand  of  attending  to  each  of  three  cueing 
conditions:  no-dot,  one-dot,  and  two-dot.  The  results  from 

six  subjects  indicate  that  attending  to  the  two-dot  cue  is 
more  difficult  than  attending  to  the  one-dot  cue  at  the 


fastest  presentation  durations.  Accuracy  was  significantly 
higher  in  the  no-dot  condition  except  at  the  slowest 
presentation  duration.  In  the  one-  and  two-dot  conditions, 
accuracy  increased  as  the  presentation  duration  increased;  at 
133  ms,  the  average  percentage  of  correct  identifications  was 
approximately  94%. 

Experiment  2  evaluated  the  effectiveness  of  the  one-  and 
two-dot  cues  in  a  target  acquisition  task.  For  the  10 
subjects  in  Experiment  2,  the  one-dot  cue  was  more  effective 
than  the  two-dot  cue  at  presentation  durations  of  less  than 
267  ms.  As  in  Experiment  1,  accuracy  increased  as  the 
presentation  duration  increased.  Performance  gradually 
improved  for  both  the  one-  and  the  two-dot  cues  during  the 
first  768  trials.  There  was  no  significant  improvement  for 
either  cue  between  768  and  1,280  trials. 

In  Experiment  3,  the  same  10  subjects  who  participated 
in  Experiment  2  continued  to  perform  the  target  acquisition 
task,  but  on  one-half  of  the  trials  the  cueing  dots  flashed 
to  simulate  an  IHADSS  boresight  requirement.  Accuracy  with 
both  the  one-  and  two-dot  cues  was  significantly  degraded  by 
the  presence  of  flashing  dots.  The  subjects  showed  no 
significant  improvement  in  accuracy  across  trials. 

The  results  of  the  three  experiments  indicate  that  (a) 
the  two  cueing  methods  (one-  and  two-dot)  are  differentially 
effective  in  cueing  shifts  of  visual  attention  at  short 
presentation  durations,  (b)  the  ability  to  use  the  cues 
improves  with  practice,  and  (c)  the  secondary  purpose  of  the 
cueing  dots  significantly  interferes  with  their  primary 
purpose.  Furthermore,  the  ability  of  the  selective  visual 
attention  paradigm  to  detect  these  effects  indicates  that  it 
is  an  appropriate  method  for  evaluating  some  aspects  of 
existing  and  proposed  aircraft  display  symbology  formats. 

At  the  end  of  the  contract  year,  a  report  describing  the 
literature  in  selective  visual  attention  and  the  three 
experiments  evaluating  the  cued  LoS  was  drafted  and  submitted 
for  internal  review. 


Work  Projected 

The  draft  report  for  this  project  will  be  revised  and 
submitted  to  ARIARDA  early  in  the  following  contract  year. 

If  directed  by  ARIARDA,  fur-.her  research  will  be  conducted  to 
evaluate  other  symbols  in  the  PNVS  symbology  set. 
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HUMAN  FACTORS  DESIGN  FOR 
MAINTENANCE  OF  ARMY  AVIATION  SYSTEMS 

Dr.  John  W.  Ruffner,  Project  Director 


Background 

Increasingly  complex  aviation  systems  are  being  devel¬ 
oped  to  enhance  the  ability  of  Army  aviators  to  fight  and 
survive  on  the  modern  battlefield.  However,  these  systems 
are  often  designed  with  little  regard  for  the  mental  and 
physical  capabilities  and  limitations  of  the  soldiers  who  are 
required  to  operate  and  maintain  them. 

Some  progress  has  been  made  in  designing  aviation 
systems  to  be  consistent  with  the  capabilities  and  limita¬ 
tions  of  the  operator;  however,  comparatively  little  atten¬ 
tion  has  been  paid  to  designing  systems  to  improve  their 
maintainability.  Traditionally,  the  maintainability  of  a 
system  is  given  the  lowest  priority  during  the  design 
process,  with  maintainability  being  secondary  to  performance, 
cost,  and  operability  criteria.  By  the  time  maintainability 
problems  are  identified,  changes  for  the  sake  of  efficient 
maintenance  are  often  not  feasible.  Furthermore,  efforts  to 
increase  system  performance  and  operability  often  result  in 
added  system  complexity,  with  a  concomitant  increase  in 
maintenance  requirements  (Bond,  1987) . 

Maintenance  costs  are  often  the  most  important  element 
in  the  life-cycle  cost  estimates  for  an  aviation  system.  A 
typical  breakdown  is  approximately  15%  for  design,  35%  for 
production,  and  50%  for  operation  and  support.  The  propor¬ 
tion  of  the  Department  of  Defense  annual  budget  required  for 
maintenance  has  been  estimated  to  be  between  25%  and  30%. 
Furthermore,  the  total  maintenance  costs  of  a  piece  of 
equipment  throughout  its  life  cycle  are  often  expected  to 
exceed  its  acquisition  costs. 

This  situation  is  exacerbated  by  the  projected  shortfall 
in  the  number  of  military-age  individuals  who  will  be  avail¬ 
able  for  maintaining  complex  aviation  systems.  The  personnel 
who  will  be  recruited  into  the  military  services  in  the  next 
20  years  will  be  fewer  in  number  and  of  lesser  aptitudes  and 
capabilities  than  at  present.  The  military  services  will 
have  to  compete  with  the  civilian  job  market  for  the  most 
capable  individuals.  In  addition,  only  a  small  percentage  of 
military  aviation  technicians  serve  more  than  a  4-year 
enlistment  period.  This  makes  it  extremely  difficult  for  the 
majority  of  maintenance  technicians  to  achieve  the  skill 
level  required  to  maintain  complex  aviation  systems. 
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Smith,  Westland  and  Crawford  (1970)  identified  three 
potential  solutions  to  the  problem  of  adequately  maintaining 
complex  military  systems:  (a)  improve  technician  skills 
through  training,  (b)  improve  job  performance  (e.g.,  trouble¬ 
shooting  aids),  and  (c)  improve  equipment  design.  They 
argued,  however,  that  efforts  to  provide  better  training  were 
not  succeeding  in  reducing  the  maintenance  problem.  In 
addition,  there  had  been  negligible  increases  in  the 
effectiveness  of  job  performance  aids. 

Smith  et  al .  (1970),  among  others,  emphasize  the  need  to 

improve  maintainability  by  influencing  the  design  of  systems 
as  early  as  possible  during  the  acquisition  process.  They 
argue  that  equipment  design  is  the  most  important  factor 
contributing  to  the  level  of  maintainability  and  that  there 
is  a  pressing  need  for  data,  methods,  and  models  that  specify 
human  factors  inputs  to  the  engineer  during  system 
development . 

More  recently,  the  Department  of  Defense  MIL-STD-470A 
(1983)  stated  that  manpower  and  personnel  shortages  are  of 
such  magnitude  that  the  maintainability  problem  must  be 
approached  through  the  design  process  as  well  as  through  the 
more  traditional  approaches  of  improving  training  and  pro¬ 
viding  job  performance  aids.  One  of  the  primary  objectives 
of  the  Army's  Manpower  and  Personnel  Integration  (MANPRINT) 
program  is  to  influence  the  design  of  military  systems  so 
that  they  can  be  operated  and  maintained  in  the  safest  and 
most  cost-effective  manner  consistent  with  the  manpower 
structure,  personnel  aptitudes  and  skills,  and  training 
resource  constraints  of  the  Army  (Department  of  the  Army, 

1987)  . 


Need 

In  response  to  the  Army's  MANPRINT  initiative,  several 
procedural  methodologies  and  models  for  applying  knowledge 
about  the  capabilities  and  limitations  of  human  operators  and 
maintainers  to  the  design  of  military  systems  have  been 
developed  or  modified.  The  majority  of  this  work  has  been 
directed  toward  the  role  of  the  human  as  system  operator 
rather  than  as  system  maintainer.  Methodologies  and  models 
are  also  needed  that  can  be  applied  toward  improving  the 
maintainability  of  aviation  systems  as  early  as  possible  in 
the  system  design  and  acquisition  processes.  In  December 
1987,  the  Army  Research  Institute  Aviation  Research  and 
Development  Activity  (ARIARDA)  requested  that  Anacapa 
Sciences  initiate  a  program  of  research  to  address  human 
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performance  problems  in  the  area  of  Army  aviation 
maintenance . 


Research  Objectives 

The  objective  of  this  research  area  is  to  identify  or 
develop  human  factors  methodologies  and  models  that  might  be 
used  to  improve  the  maintainability  design  of  emerging  Army 
aviation  systems. 


Research  Approach 

The  first  project  in  this  research  area  is  to  review  the 
literature  on  Army  aviation  maintenance,  maintainability 
design,  and  human  factors  methodologies  and  models  that  might 
be  used  to  improve  maintainability  design.  The  second 
project  is  to  conduct  a  maintenance  task  analysis  of  selected 
Army  aviation  systems.  The  task  analysis  is  a  precursor  to 
the  development  of  a  model  of  maintainer  performance  and 
workload . 


Work  Completed 

Survey  of  Methodologies  and  Models 

Beginning  in  December  1987,  approximately  130  documents 
related  to  maintenance  design  were  identified  and  reviewed. 
The  documents  include  maintainability  engineering  textbooks, 
maintainability  guidelines,  military  standards  and  handbooks, 
Army  regulations,  field  manuals,  technical  manuals,  litera¬ 
ture  reviews,  technical  reports,  journal  articles,  and 
professional  papers.  Ninety-nine  of  the  documents  reviewed 
were  retained  for  citation  in  the  project  report. 

Three  comparability  methodologies  and  seven  behavioral 
simulation  models  (both  operator  models  and  maintainer 
models)  were  identified  that  have  potential  for  improving  the 
maintainability  design  of  emerging  Army  aviation  systems. 

The  literature  on  each  methodology  and  model  was  reviewed  to 
determine  its  relative  utility  for  improving  maintainability 
design . 

Summary  of  results.  All  of  the  methodologies  and  models 
reviewed  were  judged  to  have  some  utility  for  improving  the 
maintainability  design  of  emerging  Army  aviation  systems. 
However,  the  utility  of  the  three  comparability  methodologies 
is  limited  because  they  (a)  produce  little  direct  insight 
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about  the  design  changes  required  to  improve  maintainability, 
(b)  tend  to  perpetuate  poor  maintainability  design  features 
and  personnel  practices,  (c)  rely  extensively  on  expert  judg¬ 
ment,  and  (d)  are  very  time  consuming  and  labor  intensive. 
Because  of  these  problems,  the  comparability  methodologies 
are  not  recommended  for  improving  maintainability  design. 

The  utility  of  the  four  operator  models  is  limited 
because  they  do  not  account  for  many  of  the  characteristics 
that  distinguish  system  maintenance  from  system  operation. 

In  addition,  each  of  the  models  has  specific  deficiencies 
that  limit  its  utility  for  simulating  maintainer  performance. 
Therefore,  the  operator  models,  in  their  present  forms,  are 
not  recommended  to  simulate  maintenance  performance  and  to 
predict  maintainer  workload. 

The  Maintenance  Personnel  Performance  Simulation  (MAPPS) 
model  was  developed  to  simulate  maintainer  performance  in  the 
nuclear  power  plant  environment.  The  extent  to  which  the 
MAPPS  model  may  be  useful  for  improving  the  maintainability 
design  of  emerging  aviation  systems  depends  on  the  similarity 
between  maintenance  tasks  and  working  conditions  in  a  nuclear 
power  plant  and  those  in  an  Army  aviation  maintenance 
environment . 

The  Crew  Chief  and  Profile  models  were  developed  to 
simulate  maintainer  performance  in  the  two  types  of 
activities  that  require  the  majority  of  maintainers'  time: 

(a)  accessing,  removing,  and  replacing  equipment  and  (b) 
fault  detection  and  troubleshooting.  Of  the  methodologies 
and  models  reviewed.  Crew  Chief  and  Profile  were  judged  to 
have  the  greatest  potential  for  improving  the  maintainability 
design  of  emerging  Army  aviation  systems. 

Recommendations .  A  draft  report  summarizing  the  results 
of  the  literature  review  was  completed  and  submitted  to 
ARIARDA  for  formal  review  in  February,  1989.  ARIARDA 
personnel  completed  their  review  in  May,  1989.  The  draft 
report  was  revised  and  resubmitted  in  final  form  (Ruffner, 
1989)  to  ARIARDA  in  August,  1989.  In  the  report,  it  was 
recommended  that  the  Army: 

•  evaluate  the  Crew  Chief  and  Profile  models  to  deter¬ 
mine  their  utility  for  improving  the  maintainability 
design  of  Army  aviation  systems,  and 

•  evaluate  the  four  operator  models  and  the  MAPPS  model 
to  determine  the  feasibility  and  desirability  of 
modifying  them  to  simulate  the  performance  of  main¬ 
tainer  tasks  not  addressed  adequately  by  the  Crew 
Chief  or  Profile  models. 
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Furthermore,  ten  criteria,  drawn  from  the  maintenance  and 
maintainability  design  literature,  were  proposed  to  guide  the 
development  and  evaluation  of  maintenance  design  method¬ 
ologies  and  models. 

In  addition,  it  was  recommended  that  three  additional 
research  tasks  be  undertaken  to  address  maintainer  perfor¬ 
mance  issues  in  Army  aviation  systems  and  operational 
environments : 

•  conduct  maintenance  task  and  workload  analyses  for 
selected  Army  aviation  systems  and  equipment  as  a 
precursor  to  the  development  of  a  model  of  Army 
aviation  maintainer  performance  and  workload; 

•  monitor  research  and  development  efforts  to  develop 
expert  system/CAD  maintainability  design  models  that 
could  apply  to  Army  aviation  systems;  and 

•  conduct  a  comprehensive  review  of  the  literature  on 
troubleshooting  performance,  especially  as  it  applies 
to  Army  aviation  systems,  and  monitor  ongoing  research 
and  development  activities  in  this  area. 


M.a.in.t..e.nan.c.e  T.asX.  .Analysis 

After  reviewing  the  first  project  report,  ARIARDA 
requested  that  Anacapa  Sciences  initiate  a  task  analysis  of 
selected  Army  aviation  tasks.  Work  on  this  project  began  in 
March  1989  with  a  review  of  the  literature  relevant  to 
maintenance  task  analysis.  In  addition,  the  U.S.  Army 
Aviation  Systems  Command  in  St.  Louis,  Missouri,  was 
contacted  to  identify  sources  of  reliability  and  maintain¬ 
ability  information  for  fielded  and  emerging  Army  aviation 
systems.  At  the  end  of  the  current  contract  year,  the 
literature  review  was  completed  and  work  had  begun  on  the 
development  of  a  research  plan. 


Work  Projected 

Survey  of  Methodologies  and  Models 

Submission  of  the  final  report  completed  Anacapa 
Sciences'  work  on  this  project. 

Maintenance  Task  Analysis 

The  research  plan  will  be  completed  and  submitted  to 
ARIARDA  for  formal  review  during  the  first  quarter  of  the 
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next  contract  year.  After  ARIARDA  has  reviewed  and  approved 
the  plan,  data  collection  activities  will  begin  on  this 
project.  Following  the  data  collection,  the  data  will  be 
processed  and  analyzed  and  a  report  will  be  prepared  to 
document  the  project  results. 
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EVALUATION  OF  THE  AVIATION  RESOURCE  MANAGEMENT  SURVEY 

(ARMS)  CHECKLIST 

Dr,  John  W.  Ruffner,  Project  Director 


Background 

According  to  the  Army's  "total  force"  concept,  Reserve 
Component  (RC)  aviators  serving  in  the  U.S.  Army  Reserve 
(USAR)  and  the  Army  National  Guard  (ARNG)  are  required  to 
train  to  the  same  standards  and  to  maintain  the  same  level  of 
flight  proficiency  and  flight  safety  as  aviators  serving  in 
the  Active  Component.  However,  RC  aviators  must  meet  these 
requirements  with  less  resources  (e.g.,  aircraft,  training 
time,  flying  hours,  instructor  pilots)  than  Active  Component 
aviators.  Therefore,  the  individuals  who  are  responsible  for 
planning,  implementing,  and  evaluating  RC  training  must 
manage  the  available  resources  efficiently. 

One  of  the  ways  that  the  Army  helps  RC  training  managers 
achieve  efficiency  is  through  evaluation  visits  from  Aviation 
Resource  Management  Survey  (ARMS)  teams.  As  defined  by  U.S. 
Army  Forces  Command  (FORSCOM)  Regulation  350-3  (1984),  the 
ARMS  has  four  specific  objectives: 

•  to  help  commanders  identify  strengths  and  weaknesses 
in  all  aviation-related  programs; 

•  to  assess  an  aviation  support  facility's  capacity  to 
support  the  training  of  units  assigned  to  the 
facility; 

•  to  assess  the  aviation  unit's  capabilities  (a)  to 
operate  safely,  efficiently,  and  effectively  and 
(b)  to  maintain  aviation  resources  apart  from  the 
support  facility  while  accomplishing  its  mobilization 
mission;  and 

•  to  identify  problems  and  coordinate  assistance 
required  to  solve  problems  that  are  beyond  the 
facility  commander's  or  unit  commander's  sphere  of 
authority . 

The  Deputy  Chief  of  Staff  for  Training  (DCST)  in  each  of 
the  five  Continental  U.S.  Armies  (CONUSAs)  is  responsible  for 
conducting  ARMS  evaluations.  According  to  FORSCOM  Regulation 
350-3,  an  ARMS  is  to  be  conducted  at  least  once  a  year  for 
each  USAR  facility  and  unit  and  at  least  once  every  two  years 
for  each  ARNG  facility  and  unit  within  the  CONUSA. 
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Problem 


Each  CONUSA  has  its  own  procedure  for  carrying  out  the 
ARMS  evaluation  program.  There  is  a  lack  of  standardization 
across  the  CONUSAs  in  (a)  the  functional  areas  (e.g.,  safety, 
standardization,  and  training)  that  are  evaluated,  (b)  the 
procedures  used  by  the  ARMS  teams  to  assess  the  facilities 
and  units,  and  (c)  the  standards  for  acceptable  performance. 

The  First  U.S.  Army  DCST,  Aviation  Division,  has  devel¬ 
oped  a  checklist  to  be  used  by  the  ARMS  team  during  its  eval¬ 
uation  visits.  The  checklist  originally  was  published  in 
October  1983,  and  republished  in  August  1985,  as  First  Army 
Pamphlet  95-1.  The  checklist  draws  from  two  sources: 

FORSCOM  Form  14-1-R,  "Reserve  Component  Aviation  Resource 
Management  Checklist”  (1980),  and  the  U.S.  Army  Safety 
Center,  "Guide  to  Aviation  Resources  Management  for  Aircraft 
Mishap  Prevention"  (1984). 

The  First  Army  checklist  contains  670  items  divided  into 
the  following  11  major  functional  areas  of  evaluation: 

•  Aviation  Safety  Management, 

•  Facility/Unit  Operations, 

•  Aviation  Standardization  and  Training, 

•  Aircraft/Flightline  Operations, 

•  Aeromedical  Management, 

•  Aircraft  Crash  Rescue  and  Fire  Fighting, 

•  Petroleum,  Oil,  and  Lubricants, 

•  Maintenance  Management, 

•  Aviation  Armament, 

•  Aviation  Life  Support  Equipment,  and 

•  Physical  Security. 

Each  checklist  item  describes  a  specific  deficiency  that  may 
result  in  (a)  the  failure  of  a  facility  to  accomplish  its 
mission  of  supporting  its  assigned  RC  units  or  (b)  the 
failure  of  a  unit  to  accomplish  its  mobilization  combat 
mission.  The  checklist  items  were  written  by  aviation 
subject  matter  experts  who  are  knowledgeable  about  the  opera¬ 
tional  requirements  of  RC  support  facilities  in  each  of  the 
functional  areas  or  the  mobilization  mission  requirements  for 
RC  units. 

The  DCST,  First  U.S.  Army,  has  expressed  concern  about 
the  content  of  the  checklist,  the  manner  in  which  the  check¬ 
list  items  are  used  to  evaluate  RC  facilities  and  units,  and 
the  management  and  utilization  of  information  obtained  from 
ARMS  visits.  Consequently,  during  the  second  quarter  of 
fiscal  year  1985,  the  DCST  requested  that  the  U.S.  Army 
Research  Institute  Aviation  Research  and  Development  Activity 
( ARIARDA)  provide  research  support  to  evaluate  and  revise  the 
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checklist.  Anacapa  Sciences  began  work  on  the  project  on  3 
June  1985. 


Research  Objectives 

There  are  three  general  objectives  of  the  ARMS  Checklist 
research : 

•  systematically  evaluate  the  content  of  the  First  U.S. 
Army  ARMS  Checklist, 

•  develop  recommendations  for  improving  the  ARMS  check¬ 
list  and  the  procedures  used  to  administer  it,  and 

•  develop  a  computer-based  information  management  system 
for  organizing  and  analyzing  ARMS  checklist  data. 


Research  Approach 

A  preliminary  review  of  the  ARMS  Checklist  identified 
seven  general  deficiencies  in  the  checklist  and  administra¬ 
tive  procedures: 

•  The  ARMS  Checklist  is  excessively  long  and  there  are 
many  items  that  may  not  be  highly  related  to  mission 
success  . 

•  The  procedures  used  to  evaluate  checklist  items  and  to 
combine  ratings  from  the  various  functional  areas  into 
an  overall  rating  are  not  standardized. 

•  The  negatively  stated  item  format  does  not  allow  an 
inexperienced  evaluator  to  focus  on  the  specific 
subject  in  the  item  to  be  evaluated. 

•  The  items  are  not  listed  in  an  order  that  allows  an 
inexperienced  evaluator  to  proceed  through  the 
evaluation  steps  efficiently. 

•  The  items  are  not  identified  as  applying  specifically 
to  an  aviation  facility,  an  aviation  unit,  or  both. 

•  Many  items  are  too  general  to  be  associated  with 
observable  conditions  or  events. 

•  There  is  no  systematic  procedure  for  collating 
information  about  commonly  occurring  deficiencies 
observed  across  facilities  or  units. 

The  preliminary  review  led  to  the  development  of  three 
criteria  for  determining  if  an  item  should  be  retained  in  the 
checklist.  Specifically,  an  item  should  be  retained  in  the 
checklist  only  if  the  attribute  addressed  in  the  item  is 

(a)  easily  detectable  during  an  ARMS  visit  (Detectability), 

(b)  important  for  judging  the  status  of  one  of  the  functional 
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areas  (Importance) ,  and  (c)  critical  for  mission  success 
(Criticality) .  A  survey  questionnaire  was  developed  to 
assess  the  extent  to  which  the  checklist  items  meet  the  three 
criteria  for  a  support  facility  and  for  a  unit.  A  separate 
version  of  the  questionnaire  was  developed  for  each  of  the 
functional  areas.  The  respondents  for  the  questionnaire  were 
aviators  and  aviation  technicians  from  First  Army  National 
Guard  and  Army  Reserve  aviation  support  facilities  and 
aviation  units. 


Work  Completed 

Pretesting  of  the  questionnaires  was  completed  in 
November  1985.  Following  pretest  and  revision,  the  question¬ 
naires  were  mailed  to  ARNG  and  USAR  facilities  in  the  First 
Army  area.  An  average  of  23  respondents  completed  a  ques¬ 
tionnaire  in  each  functional  area.  Responses  to  the 
questionnaires  were  entered  into  a  data  base,  verified,  and 
analyzed.  Preliminary  results  of  the  data  analyses  were 
briefed  to  the  First  Army  DCST  in  June  1986,  and  to  staff 
members  of  the  Aviation  Division,  DCST,  First  U.S.  Army  in 
March  1987. 

The  results  indicate  that  the  deficiencies  described  in 
the  majority  of  the  checklist  items  are  detected  easily 
during  an  ARMS  evaluation  visit  and  are  moderately  important 
for  assessing  the  functional  areas  in  which  they  are 
presently  classified.  The  results  also  indicate  that  a 
facility  or  unit  with  the  deficiencies  described  in  the 
majority  of  the  checklist  items  could  support  most  aspects  of 
its  mission,  assuming  that  the  deficiencies  exist  in  isola¬ 
tion.  The  results  suggest  that  a  single  version  of  the 
checklist  needs  to  be  developed,  with  each  item  presented  as 
an  affirmative  question  instead  of  a  negative  statement  and 
clearly  annotated  to  indicate  whether  it  applies  to  a 
facility  or  to  a  unit. 

A  set  of  decision  rules  was  developed  to  aid  the 
military  user  in  determining  if  items  should  be  retained  in 
their  present  form,  revised,  or  deleted  from  the  checklist  on 
the  basis  of  the  item's  Detectability,  Importance,  and 
Criticality  ratings.  The  decision  rules  should  be  applied  to 
both  the  facility  and  unit  checklist  item  ratings. 

An  information  data  base  was  developed  to  summarize  the 
checklist  items'  Detectability,  Importance,  and  Criticality 
ratings  and  to  record  the  evaluations  of  RC  units  and  facili¬ 
ties  during  future  ARMS  evaluation  visits.  The  data  base  was 
designed  to  enable  the  First  Army  ARMS  team  (a)  to  select 
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items  for  retention,  revision,  or  deletion  using  the  decision 
rules,  (b)  to  reorganize  the  checklist  by  grouping  together 
items  with  similar  content  and  reference  publications,  (c)  to 
place  the  items  in  a  sequence  that  minimizes  evaluator 
effort,  and  (d)  to  utilize  the  data  obtained  from  future  ARMS 
visits  more  effectively  (e.g.,  to  identify  commonly  occurring 
deficiencies) . 

At  the  end  of  the  second  contract  year,  the  draft 
technical  report  (Ruffner  &  McAnulty,  1987)  was  formally 
reviewed  by  ARIARDA  and  returned  to  Anacapa  for  revision.  At 
the  beginning  of  the  current  contract  year,  the  report  was 
revised  and  submitted  to  ARIARDA  in  final  form.  Volume  I  of 
the  report,  describing  the  background,  method  and  results, 
was  published  in  May  1989  as  Technical  Report  835  (Ruffner  & 
McAnulty,  1989a) .  In  addition,  an  article  (Ruffner  & 
McAnulty,  1989b)  describing  the  methodology  and  results  of 
this  project  was  accepted  for  presentation  at  the  33rd  Annual 
Meeting  of  the  Human  Factors  Society. 


Work  Projected 

Completion  and  submission  of  the  draft  final  report 
completed  Anacapa' s  work  on  this  project. 
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DEVELOPMENT  AND  VALIDATION  OF  THE  NEW 
FLIGHT  APTITUDE  SELECTION  TEST  (NFAST) 

Dr.  D.  Michael  McAnulty,  Project  Director 


Background 

The  Army's  original  selection  battery,  the  Flight 
Aptitude  Selection  Test  (FAST) ,  was  developed  in  response  to 
the  unacceptably  high  attrition  rates  in  the  Army  flight 
training  program  during  the  1950s.  The  FAST  comprised  two 
overlapping  batteries,  one  for  commissioned  officer  (CO) 
applicants  and  one  for  enlisted  and  civilian  applicants  to 
the  Warrant  Officer  Candidate  (WOC)  program.  Each  battery 
yielded  a  fixed  wing  and  a  rotary  wing  aptitude  score  for 
each  applicant  (Kaplan,  1965) .  The  FAST,  implemented  in 
1966,  resulted  in  a  substantial  reduction  in  the  flight 
training  attrition  rates. 

In  1975,  the  FAST  was  revised  to  produce  a  single, 
effective  battery  with  fewer,  shorter,  and  more  reliably 
scored  subtests.  Eastman  and  McMullen  (1978)  selected  7  of 
the  12  FAST  subtests  for  retention  in  the  revised  FAST 
(RFAST) .  Subsequently,  they  analyzed  the  item  difficulties 
and  item  discrimination  coefficients  to  select  the  items  to 
be  retained  in  each  subtest.  The  RFAST,  implemented  in  1980, 
was  approximately  one-half  the  length  of  the  original  FAST. 

Subsequent  research,  however,  indicated  the  need  to 
develop  a  new  FAST  (NFAST)  battery.  Lockwood  and  Shipley 
(1984)  found  that  six  of  the  seven  subtests  had  adequate 
internal  consistency  and  that  the  correlation  between  the 
RFAST  score  and  performance  in  initial  entry  rotary  wing 
(IERW)  training  was  statistically  significant.  They  con¬ 
cluded,  however,  that  the  low  percentage  of  variance 
accounted  for  by  the  RFAST  indicates  that  it  has  limited 
utility  in  predicting  IERW  performance.  In  addition.  Smith 
and  McAnulty  (1985)  found  that  the  RFAST  had  marginal  retest 
reliability  and  that  there  was  a  large  increase  in  the 
average  score  on  retesting,  indicating  a  need  for  an 
equivalent  form  for  use  when  retesting  is  required. 

Finally,  Oosterhof  and  Dohme  (1984)  identified  several 
problems  with  the  RFAST,  including  biased  items,  poor 
graphics  quality,  and  the  lack  of  an  alternate  form  for 
retesting.  Oosterhof  and  Dohme  developed  an  alternate  FAST 
to  remedy  the  problems  they  had  identified,  but  they  did  not 
develop  any  new  tests  for  the  selection  battery. 
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Early  in  the  development  of  the  NFAST  battery,  research 
was  conducted  to  identify  the  ability  requirements  for  the 
successful  completion  of  IERW  training.  Experienced  IERW 
instructor  pilots  (IPs)  were  asked  (a)  to  identify  the  tasks 
that  are  most  indicative  of  successful  performance  in  the 
primary  and  instrument  phases  of  IERW  and  (b)  to  judge  the 
type  and  importance  of  the  abilities  that  are  required  to 
perform  each  task.  The  task-ability  ratings  for  each  IP  were 
then  transformed  to  a  normally  distributed,  equal-interval 
scale  using  the  method  of  successive  intervals  (McAnulty  & 
Jones,  1984).  Analyses  of  the  transformed  ratings  indicated 
that  24  abilities  in  the  psychomotor,  perceptual,  language, 
and  cognitive  domains  were  required  for  successful  perfor¬ 
mance  in  IERW.  These  analyses  were  used  to  design  a  test 
specifications  matrix  to  guide  the  development  of  the  NFAST 
(McAnulty,  Jones,  Cohen,  &  Lockwood,  1984) . 

Nine  new  tests  were  developed  for  an  experimental  NFAST 
battery.  Eight  tests  were  each  designed  to  measure  a  unique 
ability  and  one  test  was  designed  to  measure  a  complex  of 
abilities  required  for  the  successful  completion  of  IERW 
training.  The  battery  also  included  four  standardized  tests 
as  marker  variables.  The  7-hour  experimental  battery  was 
administered  to  290  general  population  subjects  at  three 
military  installations  in  the  southeastern  United  States. 

The  results  indicate  that  the  complex  ability  test  and 
six  of  the  unique  ability  te'-ts  assess  reliable  individual 
differences  in  the  abilities  of  interest  (McAnulty,  Cross,  & 
Jones,  1986) .  The  average  difficulty  levels  of  the  seven 
tests  are  near  the  optimum  level  of  .50;  the  test  variances 
indicate  the  measurement  of  substantial  individual  differ¬ 
ences;  and  the  estimates  of  reliability  are  acceptable  when 
test  length  and  the  design  specifications  are  considered. 

The  remaining  two  unique  ability  tests  had  undesirable 
psychometric  characteristics  or  did  not  contribute  any  unique 
variance  to  the  factor  structure  of  the  battery. 


Need 

The  U.S.  Army  Research  Institute  Aviation  Research  and 
Development  Activity  (ARIARDA)  has  a  continuing  requirement 
to  evaluate  and  improve  the  tests  that  are  used  to  select 
applicants  for  the  Army  IERW  training  program.  As  indicated 
in  the  Background  section,  an  NFAST  battery  is  needed  to 
improve  the  reliability  and  validity  of  the  IERW  selection 
process  and  to  provide  an  equivalent  form  to  be  used  for 
retesting  IERW  applicants. 
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Research  Objective 


The  general  objective  of  this  research  area  is  to 
develop,  evaluate,  and  implement  a  more  effective  battery  of 
IERW  selection  tests.  To  accomplish  this  objective,  the 
following  research  and  development  tasks  must  be  performed: 

•  develop  two  alternate  forms  of  the  NFAST  battery, 

•  conduct  preoperational  research  to  validate  and  equate 
the  alternate  forms  of  the  NFAST  battery, 

•  produce  and  pretest  the  operational  versions  of  the 
NFAST  battery,  and 

•  implement  and  evaluate  the  NFAST  battery  and 
administrative  procedures  in  operational  use. 


Research  Approach 

This  research  is  part  of  the  ongoing  ARIARDA  program  in 
aviator  selection  and  classification  (McAnulty,  1986)  .  The 
aviator  selection  research  is  divided  into  three  projects. 

The  initial  project  (NFAST  Validation)  is  a  predictive  valid¬ 
ity  investigation.  The  results  of  the  experimental  battery 
analyses  will  be  used  to  develop  two  alternate  forms  of  an 
NFAST  validation  battery.  The  preoperational  validation 
research  will  be  conducted  (a)  to  determine  the  relationship 
between  the  NFAST  tests,  other  predictor  data,  and  performance 
in  IERW  training  and  (b)  to  equate  the  alternate  forms  of  the 
battery  on  a  large  sample  drawn  from  the  target  population  of 
flight  students.  During  this  project,  measures  of  IERW 
performance  will  be  identified  and  evaluated  as  flight 
training  criteria. 

The  second  project  (Operational  NFAST  Development)  is 
the  development  of  the  operational  NFAST  battery.  The 
results  of  the  validation  analyses  will  be  used  to  produce 
two  alternate  forms  of  the  NFAST.  In  addition,  all  ancillary 
materials  (machine  scorable  answer  sheets,  administrative 
manuals,  scoring  and  equating  manuals,  selection  criteria) 
will  be  prepared  for  operational  use.  The  operational 
battery  and  ancillary  materials  will  be  pretested  on  a  sample 
of  current  IERW  students. 

The  third  project  (NFAST  Implementation)  is  the  imple¬ 
mentation  of  the  NFAST  to  select  applicants  for  IERW 
training.  When  the  NFAST  is  implemented,  a  sample  of  field 
data  will  be  analyzed  to  ensure  that  the  psychometric  charac¬ 
teristics  of  the  operational  battery  are  not  significantly 
different  from  the  validation  battery.  In  addition,  data 
will  be  collected  and  analyzed  to  ensure  that  the  tests  are 
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being  administered  properly  and  that  the  test  scores  are 
being  used  appropriately  in  the  selection  process. 


Work  Completed 

Preope rational  NFAST  Validation 

The  validation  battery  development,  data  collection,  and 
analysis  activities  on  the  NFAST  Validation  project  have  been 
completed.  The  results  of  the  experimental  battery  research 
were  used  to  develop  two  alternate  forms  of  the  NFAST  valida¬ 
tion  battery.  Each  form  consists  of  modified  versions  of  the 
complex  test  and  the  six  unique  ability  tests  that  had 
acceptable  psychometric  characteristics.  In  general,  the 
validation  battery  tests  are  approximately  two-thirds  the 
length  of  the  experimental  battery  tests.  The  alternate 
forms  of  five  of  the  tests  have  40%  to  50%  of  the  items  in 
common.  The  complex  ability  test  forms  and  one  of  the  unique 
ability  test  forms  do  not  have  any  identical  items.  Finally, 
a  knowledge  test  of  helicopter  operations  and  aerodynamic 
principles  was  adapted  from  the  RFAST  battery  for  inclusion 
in  the  validation  battery.  The  items  on  the  knowledge  test 
are  identical  on  both  forms.  Each  form  of  the  validation 
battery  requires  approximately  4  hours  to  administer. 

Between  March  and  October  1987,  the  alternate  forms  of 
the  NFAST  battery  were  administered  to  approximately  97%  of 
the  CO  and  WOC  flight  students  during  their  first  week  of 
IERW  training.  When  the  test  administration  segment  was 
terminated,  complete  and  usable  test  data  were  collected  from 
377  CO  and  341  WOC  students. 

Test  results.  Analyses  of  the  test  data  indicate  that 
the  flight  student  performance  on  the  validation  battery, 
excluding  the  helicopter  knowledge  test,  is  similar  to  the 
general  population  performance  on  the  experimental  battery. 
The  average  difficulty  levels  are  near  .50  despite  the  more 
restrictive  time  limits  that  were  imposed  on  the  validation 
tests,  and  the  variances  indicate  that  substantial  individual 
differences  in  ability  are  being  measured  by  the  tests.  The 
internal  consistency  estimates  of  reliability  are  also  within 
an  acceptable  range.  Performance  on  the  two  forms  of  the 
battery  is  very  similar  except  for  one  of  the  unique  ability 
tests.  Test  performance  by  the  CO  and  WOC  students  is  also 
quite  similar,  although  the  CO  students  scored  significantly 
higher  (p.  <  .01)  on  four  of  the  tests. 

The  results  on  the  helicopter  knowledge  test  indicate 
that  the  test  is  not  difficult  and  that  there  is  limited 
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variability  in  the  scores.  WOC  students  scored  significantly 
higher  on  the  test  than  the  CO  students.  However,  there  is 
no  difference  in  performance  by  either  student  group  on  the 
two  forms  of  the  test.  Because  the  two  forms  are  identical, 
this  result  indicates  there  was  no  systematic  sampling  bias 
in  terms  of  aviation-related  knowledge  in  assigning  students 
to  the  alternate  forms  of  the  NFAST  battery. 

Training  results.  Three  types  of  IERW  performance 
measures  were  collected  on  the  students  who  participated  in 
the  NFAST  validation:  administrative  changes  (elimination 
and  training  setback  data) ,  flight  hour  data  (number  of 
flight  hours  required  to  complete  each  phase  of  training) , 
and  IERW  training  grades  (academic  and  flight) .  The  primary 
performance  criterion  was  the  Overall  Average  Grade  (OAG) , 
which  is  a  weighted  composite  of  the  academic  and  flight 
grades.  IERW  performance  data  collection  was  completed  in 
January  1989.  Of  the  718  students  in  the  data  base,  696 
either  completed  IERW  or  were  eliminated  from  training  for 
flight  or  academic  deficiencies;  the  remaining  students  were 
eliminated  for  nondeficiency  reasons  or  were  transferred  for 
training  under  a  different  syllabus.  The  validation  analyses 
were  conducted  using  the  696  students  who  graduated  or  were 
eliminated  for  deficiency  reasons. 

Validation  results.  The  results  of  the  validation  data 
analyses  indicate  (a)  that  the  NFAST  battery  has  good  psycho¬ 
metric  characteristics,  (b)  that  the  alternate  forms  are 
approximately  equal,  and  (c)  that  a  subset  of  the  tests  in 
the  battery  will  significantly  improve  the  IERW  selection 
process.  Across  all  subgroups  (e.g.  forms,  ranks,  training 
track,  education  levels),  three  of  the  tests  (the  complex 
test,  one  unique  ability  test,  and  the  helicopter  knowledge 
test)  consistently  had  high  regression  coefficients  (£  of 
approximately  .50)  on  the  OAG  and  other  criteria.  Cross 
validation  analyses  indicated  that  the  regression  equations 
were  stable  when  applied  to  the  holdout  samples.  Finally, 
utility  analyses  demonstrated  the  improvements  in  training 
efficiency  that  could  be  obtained  by  using  shortened  versions 
of  the  three  tests  as  the  IERW  selection  battery. 

In  December  1988,  the  results  of  the  preoperat ional 
validation  research  were  presented  at  the  annual  conference 
of  the  Military  Testing  Association  (McAnulty,  1988) .  At  the 
end  of  the  current  contract  year,  a  draft  report  was  being 
prepared  to  document  the  results  of  the  NFAST  Validation 
project . 
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The  results  of  the  validation  research  were  used  to 
modify  the  validation  battery  for  operational  use.  The 
modifications  included  shortening  the  test  length,  changing 
items  with  poor  characteristics,  revising  the  test  instruc¬ 
tions,  and  improving  the  graphics  and  format  of  the  tests. 
In  addition,  one  of  the  RFAST  tests  (the  Cyclic  Orientation 
Test)  was  selected  for  retention  in  the  operational  NFAST 
battery . 

New  ancillary  materials  were  also  developed  for  the 
operational  battery.  A  test  administration  manual  was 
drafted  and  a  new  answer  sheet  was  designed  for  the  NFAST. 
The  answer  sheet  requests  additional  biographic  information 
that  the  validation  analyses  indicated  was  related  to 
successful  performance  in  IERW. 


Work  Projected 

Preope rational  NFAST  Validation 

The  preparation  and  revision  of  the  report  documenting 
the  NFAST  validation  research  will  be  completed  by  December 
1989.  Submission  of  this  report  to  ARIARDA  will  complete  the 
scheduled  activities  on  this  project. 


Operational  NFAST  Development 

A  pretest  of  the  operational  battery,  including  the  test 
retained  from  the  RFAST,  will  be  conducted  with  approximately 
200  entering  IERW  students.  The  pretest  data  will  be  used  to 
evaluate  the  modifications  that  were  made  to  the  validation 
battery  prior  to  implementing  the  NFAST.  In  addition,  a 
scoring  manual  and  an  NFAST  information  pamphlet  will  be 
prepared.  Preparation  of  the  operational  materials  will  be 
completed  in  January  1990. 


NFAST  Implementation 

When  the  NFAST  pretesting  has  been  completed,  the  opera¬ 
tional  batteries  and  ancillary  materials  will  be  delivered  to 
the  U.S.  Army  Soldier  Support  Center  for  review,  reproduc¬ 
tion,  and  implementation.  After  the  NFAST  is  in  operational 
use,  follow-on  research  will  be  conducted  to  ensure  that 
applicant  performance  on  the  test  batteries  is  within  accept¬ 
able  limits,  that  administrative  procedures  are  being 


followed,  and  that  the  selection  criteria  are  valid. 
Depending  on  the  results  of  the  follow-on  research,  a  second 
validation  investigation  will  be  conducted,  if  necessary, 
using  an  unrestricted  sample  (i.e.,  not  already  selected  for 
flight  training)  of  IERW  applicants. 
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DEVELOPMENT  OF  A  PEER  COMPARISON  PROGRAM 
Dr.  D.  Michael  McAnulty,  Project  Director 


Background 

This  project  was  initiated  to  assist  the  School 
Secretary,  U.S.  Army  Aviation  Center,  Fort  Rucker,  Alabama, 
in  developing  an  algorithm  to  select  course  honor  graduates 
based  on  the  "whole  person"  concept.  The  School  Secretary 
wanted  to  augment  the  academic  grade  criterion  used  to  select 
honor  graduates  in  the  Aviation  Officer  Advanced  Course 
(AVNOAC) ,  a  5-month  training  course  for  captains  and  pro- 
motable  first  lieutenants.  The  two  purposes  of  the  augmented 
program  are  (a)  to  motivate  students  to  maximize  their 
military  and  academic  efforts  during  the  course  and  (b)  to 
identify  students  who  have  high  potential  as  Army  aviation 
officers  at  an  early  stage  of  their  careers. 

Specifically,  the  School  Secretary  was  interested  in 
using  peer  assessments  by  the  AVNOAC  students  as  a  component 
in  the  honor  graduate  selection  algorithm.  The  peer  assess¬ 
ments  were  to  evaluate  aspects  of  the  students’  performance 
that  were  not  reflected  in  their  academic  scores.  Instructor 
ratinas  were  not  considered  as  a  criterion  component  because 
of  the  limited  interaction  between  the  school  cadre  and  the 
students . 


Research  Objectives 

Following  a  review  of  the  peer  assessment  literature  and 
the  AVNOAC  syllabus,  a  peer  comparison  (PC)  methodology  was 
proposed  for  use  in  the  AVNOAC.  The  School  Secretary  agreed 
to  support  the  following  research  activities: 

•  identify  the  most  important  military  qualities  that 
can  be  assessed  by  peers  during  the  AVNOAC, 

•  develop  the  PC  instruments  and  procedures  for  use  in 
the  AVNOAC,  and 

•  evaluate  the  PC  technique  prior  to  its  implementation. 

Research  Approach 

The  research  approach  was  divided  into  three  phases  that 
correspond  to  the  research  objectives.  During  Phase  1,  a 
military  qualities  survey  was  administered  to  identify  the  most 
important  qualities  that  can  be  assessed  by  peers  during  the 
AVNOAC.  The  survey  asked  senior  aviation  officers  to  rate  a 
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list  of  primary  military  qualities  as  potential  dimensions  for 
evaluating  student  performance  and  for  identifying  students 
with  high  career  potential.  The  survey  data  provided  the 
information  needed  to  develop  the  PC  instruments. 

During  Phase  2,  three  project  assessment  instruments 
were  developed:  the  PC  form  to  be  completed  by  the  class 
members  to  evaluate  their  peers,  a  faculty  advisor  rating 
(FAR)  form  to  be  completed  by  each  class  member's  training 
officer,  and  a  student  critique  to  be  completed  by  the 
students  to  evaluate  the  PC  instruments  and  procedures. 

During  Phase  3,  the  PC  technique  was  administered  on  an 
experimental  basis  to  two  AVNOAC  classes  and  evaluated  for 
its  potential  utility  as  an  operational  component  of  the 
honor  graduate  algorithm. 


Work  Completed 

Phase  1:  Military  Qualities  Survey 

Following  a  search  of  the  literature  and  a  review  of 
current  Army  student  evaluation  dimensions,  definitions  of  14 
primary  military  qualities  (e.g.,  adaptability,  initiative, 
judgment,  leadership,  and  responsibility)  were  compiled  for 
evaluation  by  senior  aviation  officers.  Several  important 
military  qualities  were  excluded  from  the  survey  because  they 
are  evaluated  by  academic  scores  or  are  unlikely  to  be  demon¬ 
strated  during  the  AVNOAC.  Sixteen  senior  Army  aviation 
officers  were  asked  to  rate  each  quality  on  the  following 
four  scales: 

•  importance  to  the  performance  of  captains, 

•  importance  to  the  performance  of  senior  officers, 

•  probability  of  demonstration  during  the  AVNOAC,  and 

•  degree  of  overlap  with  the  other  qualities. 

Eleven  surveys  were  completed  and  returned.  Three  of 
the  qualities  (leadership,  judgment,  and  responsibility)  had 
consistently  high  ratings  and  were  selected  as  PC  dimensions. 
Seven  of  the  qualities  were  clearly  perceived  as  being 
inappropriate  PC  dimensions.  Appearance  and  cooperation  were 
selected  from  the  remaining  four  qualities  as  two  additional 
PC  dimensions. 


Phase  2:  Form  Development 

Three  research  forms  were  developed  for  use  in  this 
project.  The  PC  form  was  developed  from  (a)  the  results  of 
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the  military  qualities  survey,  (b)  a  combination  of  the  peer 
nomination  and  peer  ranking  techniques  (e.g.,  Kane  &  Lawler, 
1978) ,  and  (c)  the  psychophysical  method  of  paired  compar¬ 
isons  (Engen,  1971) .  On  the  PC  form,  each  section  member 
(classes  are  divided  into  two  sections)  is  required  to 
nominate  and  rank  order  five  peers  on  the  basis  of  their 
potential  as  Army  aviation  officers.  The  section  member  then 
makes  paired  comparisons  of  the  nominees  on  the  five  military 
qualities  that  were  selected  from  the  military  qualities 
survey . 

PC  scores  are  computed  for  each  peer  by  first  summing 
the  rank  score  (five  points  for  first  rank,  four  points  for 
second  rank,  .  ..,  one  point  for  fifth  rank)  from  each  nomi¬ 
nating  section  member.  The  summed  rank  scores  are  then  added 
to  the  number  of  favorable  comparisons  the  peer  received  on 
each  military  quality.  Finally,  the  total  is  divided  by  the 
maximum  possible  score  to  enable  direct  comparisons  betw^e^ 
sections  with  unequal  numbers  of  students.  The  PC  scores  can 
range  from  0.0  (no  nominations)  to  1.0  (ranked  first  by  all 
section  members  and  always  favorably  compared  with  the  other 
nominees) .  Because  each  section  member  nominates  five  peers 
out  of  approximately  50  students,  a  PC  score  greater  than  .20 
probably  represents  a  consensus  among  the  section  members 
that  the  student  has  high  potential  as  an  Army  aviation 
officer . 

A  FAR  form  was  developed  to  obtain  independent  evalua¬ 
tions  of  the  students'  potential  as  Army  aviation  officers. 
Each  AVNOAC  faculty  advisor  usually  supervises  six  or  seven 
students.  On  the  FAR,  the  advisors  estimate  the  officer 
potential  of  their  students  by  assigning  them  percentile 
ranks  in  an  average  group  of  100  captains. 

Finally,  a  student  critique  form  was  developed  to  ascer¬ 
tain  student  attitudes  toward  the  peer  comparison  program. 

The  students  are  asked  to  rate  the  fairness,  utility,  aver¬ 
siveness,  and  difficulty  of  various  aspects  of  the  program. 
They  are  also  asked  to  express  their  opinions  about  the 
implementation  of  the  program  and  to  offer  recommendations 
for  improving  the  program. 

Phase  3:  Experimental  Evaluations 

First  administration.  Peer  comparisons  were  collected 
on  an  experimental  basis  (i.e.,  the  PC  scores  were  not  used 
to  select  honor  graduates)  from  Sections  1  (n  =  38)  and  2 
(H  =  40)  of  AVNOAC  85-2  during  the  fourth  month  of  training. 

A  second  set  of  PC  ratings  and  the  student  critiques  were 
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collected  from  Section  1  (n  =  33)  and  Section  2  (n  =  28)  at 
the  end  of  the  course,  approximately  1  month  later.  The 
faculty  advisors  completed  the  FARs  immediately  after 
graduation.  In  addition,  the  final  academic  averages  (AVGs) 
were  obtained  from  the  School  Secretary's  office. 

The  scores  for  the  first  and  second  data  collections 
were  highly  correlated  (Section  1  =  .96  and  Section  2  =  .86), 
indicating  the  stability  of  the  ratings  over  time.  Because 
of  the  high  correlations,  the  ratings  from  the  two  data 
collections  were  combined  into  a  single  PC  score  for  each 
peer.  The  PC  scores  ranged  from  .00  to  .48  in  Section  1  and 
from  .00  to  .36  in  Section  2.  Four  peers  in  Section  1  and 
three  peers  in  Section  2  received  PC  scores  greater  than  .20. 
A  majority  of  the  PC  scores  in  both  sections  were  between  .00 
and  .05.  The  scores  indicate  a  consensus  among  the  members 
of  the  class  in  identifying  peers  with  the  highest  potential 
as  aviation  officers. 

For  Sections  1  and  2,  respectively,  the  PC  correlations 
are  .45  and  .33  with  the  FAR,  and  .55  and  .30  with  the  AVG . 
These  moderate  correlations  show  an  expected  relationship 
between  evaluations  of  the  same  individuals  but  indicate  that 
the  PC  score  is  measuring  a  unique  perspective  of  the  class 
members.  The  correlations  between  the  FAR  and  AVG  are  .76 
and  .59  in  Sections  1  and  2,  respectively.  The  higher  FAR- 
AVG  correlations  probably  indicate  that  the  faculty  advisors 
depended  on  the  academic  average  as  a  primary  source  of 
information  in  making  their  ratings. 

Finally,  the  responses  to  the  PC  critique  were  tabu¬ 
lated.  The  overall  reaction  of  the  class  members  to  the  PC 
program  was  negative:  a  majority  indicated  that  the  PC  was 
very  biased,  slightly  or  not  at  all  useful,  and  slightly  or 
not  at  all  predictive  of  future  performance.  Furthermore, 

72%  of  the  respondents  were  either  very  or  extremely  unfavor¬ 
able  toward  implementing  the  program.  The  responses  to  the 
other  critique  items  reflected  combinations  of  positive, 
negative,  and  neutral  attitudes,  without  any  attitude 
representing  a  majority  opinion. 

The  results  of  the  first  administration  indicated  that 
the  PC  technique  was  a  potentially  useful  procedure  for 
identifying  the  class  members  with  the  highest  potential  as 
Army  aviation  officers,  although  the  students  were  critical 
of  its  use.  There  were,  however,  several  problems  with  the 
first  administration.  First,  the  students  were  not  advised 
about  the  PC  program  before  the  experime.  al  administration. 
Second,  a  surreptitious  attempt  by  the  class  leaders  to  eval¬ 
uate  the  section  members  was  discovered  just  before  the 
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second  data  collection.  Both  problems  may  have  affected  the 
students'  attitudes  about  class  evaluations.  Finally,  the 
period  of  time  between  the  first  and  second  PC  administra¬ 
tions  was  too  short  to  evaluate  the  stability  of  the  peer 
assessments . 

Second  administration.  The  second  experimental  adminis¬ 
tration  was  designed  as  a  replication  of  the  first  adminis¬ 
tration,  with  the  following  changes: 

•  students  were  advised  in  advance  of  the  research, 

•  other  nonacademic  evaluations  were  prohibited, 

•  3  months  elapsed  between  the  initial  and  final  data 
collection, 

•  the  military  quality  definitions  were  modified 
slightly, 

•  the  order  of  presentation  of  the  military  qualities 
and  nominee  pairs  was  completely  counterbalanced,  and 

•  a  new  roster  coding  system  was  instituted  to  protect 
student  privacy. 

Usable  PC  ratings  were  collected  from  48  students  in 
each  section  of  AVNOAC  86-1  during  the  second  month  of 
training.  At  the  end  of  the  course,  47  students  in  Section  1 
and  44  students  in  Section  2  completed  usable  PC  ratings  and 
student  critiques .  After  graduation,  FARs  were  completed  by 
most  of  the  faculty  advisors  and  the  AVGs  were  collected  from 
the  School  Secretary's  office. 

Two  types  of  reliability  coefficients  were  computed  on 
the  AVNOAC  86-1  ratings.  First,  the  correlations  between  the 
initial  and  final  ratings  are  .79  in  Section  1  and  .93  in 
Section  2,  indicating  the  stability  of  the  ratings  across  a 
period  of  approximately  3  months.  Second,  split-half  (odd- 
even)  correlations  for  each  data  collection  for  each  section 
were  computed  to  evaluate  the  internal  consistency  of  the 
ratings.  The  correlations  are  .71  and  .76  for  Section  1  and 
.93  for  both  data  collections  for  Section  2.  The  reliability 
coefficients  are  acceptable  in  all  cases,  although  they  are 
substantially  higher  in  Section  2.  Because  of  the  high 
correlations,  the  ratings  from  the  two  data  collections  were 
combined  into  a  single  PC  score  for  each  peer  in  each 
section . 

The  PC  scores  ranged  from  .00  to  .24  in  Section  1  and 
from  .00  to  .47  in  Section  2.  Four  peers  in  Section  1  and 
two  peers  in  Section  2  received  PC  scores  greater  than  .20. 
The  majority  of  the  PC  scores  in  both  sections  were  between 
.00  and  .05.  The  scores  indicate  a  consensus  among  the 
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members  of  Section  2  in  identifying  the  two  peers  (PCs  =  .42 
and  .47)  with  the  highest  potential  as  aviation  officers. 

The  PC  scores  in  Section  1  also  distinguish  the  peers  having 
the  highest  potential,  even  though  the  PC  scores  are  much 
lower.  The  lower  scores  could  be  an  artifact  of  the  method¬ 
ology  if  there  are  more  than  five  peers  with  high  potential 
who  are  not  substantially  different  from  each  other. 

The  PC  scores  were  then  correlated  with  the  FARs  and 
AVGs.  For  Sections  1  and  2,  respectively,  the  PC  correla¬ 
tions  are  .02  and  .30  with  the  FAR,  and  .24  and  .27  with  the 
AVG .  These  correlations  indicate  that  the  PC  score,  compared 
to  the  FAR  and  AVG,  is  measuring  a  different  aspect  of  the 
class  members'  performance.  The  .02  correlation  between  the 
FAR  and  PC  in  Section  1  is  partially  attributable  to  the 
highly  skewed  distribution  of  FARs.  The  correlations  between 
the  FAR  and  AVG  are  .53  in  both  sections.  The  FAR-AVG  corre¬ 
lations  probably  indicate  that  the  faculty  advisors  used  the 
academic  average  as  a  primary  source  of  rating  information. 

Finally,  the  PC  critique  responses  from  class  86-1  were 
negative  overall,  but  not  as  negative  as  those  from  class 
85-2.  A  majority  of  the  86-1  respondents  indicated  that  the 
PC  was  either  slightly  or  not  at  all  useful  for  selecting 
AVNOAC  honor  graduates;  the  respondents  were  approximately 
evenly  divided  on  the  issues  of  PC  fairness,  bias,  and 
predictability  of  future  Army  performance.  Ratings  of  the 
adequacy  of  definitions  and  the  difficulty  of  nominating, 
ranking,  and  comparing  peers  were  very  similar  to  the  results 
from  class  85-2.  Despite  the  slight  positive  shift  in 
attitude  toward  the  PC  program,  69%  of  the  respondents  were 
still  either  very  or  extremely  unfavorable  toward  any 
potential  implementation  of  the  program. 

A  report  (McAnulty,  1989)  that  describes  the  development 
and  evaluation  of  the  PC  procedure  was  written  and  submitted 
to  ARIARDA  with  the  following  conclusions  and  recommenda¬ 
tions.  The  results  of  the  AVNOAC  classes  85-2  and  86-1  data 
collections  indicate  that  the  PC  technique  is  a  potentially 
useful  procedure  for  identifying  the  peers  with  the  highest 
potential  as  Army  aviation  officers,  at  least  in  terms  of  the 
reliability  of  the  ratings.  There  was  a  consensus  about 
which  peers  had  the  highest  potential,  and  the  ratings  were 
generally  consistent  over  a  3-month  data  collection  interval. 
However,  longitudinal  research  is  required  to  determine  the 
validity  of  the  PC  technique  for  predicting  future 
performance.  The  students  in  both  AVNOAC  classes  found  the 
rating  procedure  to  be  aversive  and  were  unfavorable  toward 
implementing  the  PC  technique. 
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Work  Projected 

Submission  of  the  report  completed  all  activities  on 
this  project. 
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ARMY  AVIATION  AMMUNITION  AND  GUNNERY  SURVEY 
Dr.  D.  Michael  McAnulty,  Project  Director 


Background 

In  January  1987,  the  Department  of  Gunnery  and  Flight 
Systems  (DGFS)  at  the  U.S.  Army  Aviation  Center  (USAAVNC) , 
Fort  Rucker,  Alabama,  requested  that  the  U.S.  Army  Research 
Institute  Aviation  Research  and  Development  Activity 
(ARIARDA)  provide  research  support  in  conducting  an  ammuni¬ 
tion  and  gunnery  survey  of  active  Army  (AA)  and  National 
Guard  (NG)  aviation  units.  ARIARDA  agreed  to  assist  in 
designing  and  pretesting  the  survey;  to  develop  the  data 
entry,  verification,  and  analysis  programs;  to  conduct  the 
required  data  analyses;  and  to  prepare  briefing  materials  and 
technical  reports  as  required.  All  other  project  activities 
(e.g.,  administrative  coordination,  pretesting,  data  collec¬ 
tion,  data  entry)  were  to  remain  the  responsibility  of  the 
DGFS  Study  Group. 


Problem 

The  survey  research  was  designed  to  provide  an  empirical 
data  base  for  addressing  three  major  problem  areas.  First, 
the  increasing  cost  of  ammunition  and  the  competition  for 
Department  of  Defense  funds  have  created  pressure  to  reduce 
the  annual  allocation  of  ammunition  for  Army  aviation  gunnery 
training.  The  research  was  intended  to  document  the  current 
utilization  of  ammunition  in  aviation  gunnery  training,  to 
evaluate  the  success  of  the  gunnery  training,  and  to  compile 
estimates  of  the  amount  of  ammunition  required  to  maintain 
specified  Standards  in  Training  Commission  (STRAC)  readiness 
conditions.  In  addition  to  justifying  the  ammunition 
allocations,  the  resulting  data  base  was  used  by  DGFS  to 
develop  a  new  gunnery  training  manual. 

The  second  major  problem  is  the  lack  of  adequate  ranges 
for  training  and  qualifying  unit  aviators.  Many  units  com¬ 
plain  that  the  available  ranges  lack  the  targetry,  scoring 
devices,  and  space  required  for  effective  training.  Further¬ 
more,  limited  access  to  the  ranges  inhibits  the  gunnery 
training  and  makes  it  difficult  to  maintain  the  required 
readiness  conditions.  The  research  was  intended  to  document 
the  availability,  type,  and  utility  of  gunnery  ranges 
currently  in  use  by  Army  aviation  units. 

The  final  major  problem  is  the  lack  of  empirical  data 
about  the  utility  of  flight  simulators  for  weapon  systems 
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training.  Theoretically,  flight  simulators  can  reduce  the 
effects  of  the  first  two  problems.  That  is,  weapon  training 
can  be  conducted  without  ranges  and  without  incurring  ammuni¬ 
tion  costs.  However,  there  are  no  data  that  identify  the 
tasks  that  can  be  trained  effectively  in  simulators,  the 
amount  of  training  that  is  most  cost-effective,  or  the  extent 
that  flight  simulator  training  can  offset  the  need  for  weapon 
firing  in  the  aircraft . 

The  latter  problem  is  compounded  by  the  single  configu¬ 
ration  of  the  AH- 1  attack  helicopter  flight  and  weapons 
simulator  (FWS)  that  is  used  by  unit  aviators  who  fly 
different  configurations  of  AH-1  attack  helicopters  (e.g., 
AH-1G,  AH-1S  Modified,  and  AH-1S  Production  models) .  The  FWS 
is  configured  like  the  AH- IS  Fully  Modernized  helicopter. 

The  survey  research  was  designed  to  collect  information  about 
the  utilization  of  flight  simulators  for  conducting  aerial 
weapon  training. 


Research  Methodology 

In  January  1987,  the  Commanding  General  of  the  USAAVNC 
directed  that  a  survey  of  field  unit  aviators  and  aviation 
unit  commanders  be  conducted  to  compile  the  research  data 
required  to  meet  the  following  project  objectives: 

•  describe  the  current  attack  aviation  force, 

•  formulate  an  accurate  ammunition  procurement  request, 

•  evaluate  the  training  value  of  flight  simulators  for 
aerial  gunnery,  and 

•  support  revisions  to  the  Army's  aerial  gunnery 
training  programs. 

Survey  Development, 

Survey  development  began  with  a  review  of  the  relevant 
literature,  the  then  current  aerial  gunnery  training  manual 
(Field  Manual  (FM)  1-140),  and  a  previous  STRAC  questionnaire 
(1987).  The  DGFS  Study  Group  then  delineated  the  Essential 
Elements  of  Analysis  (EEA)  for  the  survey.  Approximately  100 
preliminary  survey  items  were  drafted  in  the  following  ten 
topics  covered  by  the  EEA: 

•  personal  data  about  the  respondent, 

•  military  experience  of  the  respondent, 

•  flight  experience  of  the  respondent, 

•  present  duty  assignment  of  the  respondent, 
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•  suitability  of  current  gunnery  training  publications, 

•  weapon  systems  on  the  aviator's  primary  aircraft, 

•  ammunition  allocated  and  fired  during  the  1987 
training  year, 

•  utilization  of  gunnery  range  facilities, 

•  utility  of  flight  simulators  for  gunnery  training  and 
qualification,  and 

•  door  gunnery  training. 

The  preliminary  survey  items  were  administered  to 
approximately  50  attack  helicopter  aviators  by  DGFS  person¬ 
nel.  The  pretest  results  were  used  to  produce  the  opera¬ 
tional  survey,  which  was  divided  into  two  forms:  Form  A  for 
the  unit  aviators  and  Form  B  for  the  unit  commanders.  Many 
of  the  items  on  the  two  forms  are  similar  in  content,  but  the 
unit  aviators  were  instructed  to  respond  to  the  items  with 
respect  to  themselves  and  the  unit  commanders  were  instructed 
to  respond  to  the  items  with  respect  to  the  entire  unit, 
except  for  their  personal  data  and  experience.  Anacapa 
Sciences  personnel  edited  the  final  versions  of  the  survey 
forms  and  prepared  the  required  ancillary  materials  (e.g., 
letters  of  instruction) .  Subsequently,  the  surveys  were 
approved  for  use  by  the  U.S.  Army  Soldier  Support  Center  and 
reproduced  for  administration  by  DGFS. 

Form  A  contains  68  items  divided  into  9  of  the  10  topic 
areas  listed  previously;  no  questions  are  posed  to  the  unit 
aviators  about  door  gunnery.  Form  B  contains  78  items 
divided  into  all  10  topic  areas.  The  surveys  are  much  more 
comprehensive  than  the  number  of  items  indicates.  That  is, 
many  items  have  multiple  sections  or  require  a  succession  of 
responses.  Although  all  the  items  do  not  apply  to  all  the 
respondents,  there  are  472  codable  responses  on  Form  A  and 
644  codable  responses  on  Form  B.  In  addition,  both  forms 
have  several  open-ended  response  items. 


Survey  Data  Collection 

During  August  1987,  DGFS  personnel  distributed  362 
commander  forms  and  1996  aviator  forms  to  the  AA  and  NG 
units.  The  majority  of  the  surveys  were  administered  by 
installation  points  of  contact.  The  remainder  of  the  surveys 
were  administered  by  DGFS  personnel  during  visits  to  field 
units  . 

ARIARDA  personnel  developed  computer  programs  to  enter 
and  verify  the  survey  data.  DGFS  and  Anacapa  personnel 
entered  and  verified  the  survey  responses  as  the  forms  were 
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received  from  the  aviators  and  commanders.  Data  collection 
was  terminated  on  19  November  1987.  At  that  time,  127  (35%) 
usable  unit  commander  forms  and  810  (41%)  usable  unit  aviator 
forms  had  been  completed  and  returned  to  DGFS  for  processing. 
In  addition,  35  commander  and  184  aviator  forms  were  returned 
either  unused  or  incomplete.  The  percentage  of  NG 
respondents  was  36.2%  for  the  unit  commanders  and  31.9%  for 
the  aviators. 

Several  meetings  were  held  with  DGFS  personnel  to 
enumerate  the  most  important  research  issues  and  to  identify 
the  subset  of  survey  items  that  address  those  issues.  Subse¬ 
quently,  ARIARDA  and  Anacapa  personnel  developed  a  statisti¬ 
cal  approach,  wrote  the  required  computer  programs,  and 
analyzed  the  data.  The  AA  and  NG  data  were  analyzed 
separately  because  of  the  major  differences  in  unit  mission, 
types  of  aircraft  flown,  and  training  resources  and 
standards . 


Results  and  Conclusions 

The  return  rates  of  usable  survey  forms  were  judged  to 
provide  a  reliable  data  base  for  analysis,  although  there 
were  limitations  on  the  number  of  subsample  analyses  that 
could  be  conducted.  The  results  of  the  survey  data  analyses 
were  documented  in  two  reports  (McAnulty,  Cross,  &  DeRoush, 
1989;  McAnulty  &  DeRoush,  1988)  and  are  summarized  in  the 
seven  general  conclusions  discussed  in  the  following 
paragraphs . 

1.  The  AA  respondents  are,  on  the  average,  relatively 
young  and  inexperienced  in  their  occupational  specialty.  The 
NG  aviators  are  older  and  more  experienced  than  their  AA 
counterparts  and,  therefore,  may  be  able  to  maintain  their 
skills  at  acceptable  levels  with  less  training  resources. 

The  AA  and  NG  commanders  have  approximately  equal  experience 
levels . 

2.  The  average  AA  aviator  flew  slightly  more  than  the 
minimum  number  of  hours  required  to  maintain  his  flight 
skills  in  Fiscal  Year  1987  (FY87),  but  fired  less  than  the 
authorized  number  of  ammunition  rounds.  The  average  NG 
aviator  flew  fewer  hours  and  fired  less  ammunition  than  his 
AA  counterpart . 

3.  A  substantial  number  of  AA  and  NG  units  were  unable 
to  meet  their  training  readiness  standards  with  the  resources 
available  to  them  in  FY87.  The  lack  of  sufficient  ammunition 
was  the  most  frequently  cited  reason  for  not  meeting  the 
standards,  but  other  resource  limitations  were  also  cited. 
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4.  Gunnery  ranges  were  not  readily  available  to  many 
units  or  did  not  have  adequate  scoring  methods.  Very  few  of 
the  ranges  were  designed  specifically  for  aviation  gunnery, 
and  most  were  shared  with  other  branches .  These  problems 
were  especially  critical  for  the  NG  units. 

5.  The  AA  aviators  used  flight  simulators  for  gunnery 
training  to  a  moderate  extent  (a  median  of  10  hours)  during 
FY87 .  Very  few  NG  units  had  access  to  simulators,  so  their 
simulator  usage  data  were  not  analyzed.  The  AA  respondents 
perceived  the  simulators  to  have  utility  for  some  types  of 
training  tasks  but  not  for  other  types.  Specifically,  tasks 
that  were  highly  dependent  on  the  simulator  visual  system 
generally  received  lower  ratings.  The  lack  of  physical 
fidelity  between  the  AH-1  models  and  the  FWS  was  not  judged 
to  impair  training  on  most  types  of  tasks.  AH-64  aviators 
rated  the  training  value  of  the  AH-64  Combat  Mission 
Simulator  higher  than  the  AH-1  aviators  rated  the  FWS  on  7  of 
12  types  of  training. 

6.  The  respondents  indicated  a  desire  for  standaruized 
gunnery  tables  to  support  the  development  of  unit  training 
programs.  The  data  obtained  from  the  survey  respondents 
constitute  a  source  of  information  for  revising  the  FM  1-140 
tables . 

7.  The  estimates  of  ammunition  requirements  indicate 
that  the  current  STRAC  authorizations  approximate  the  minimum 
number  of  rounds  needed  to  qualify  and  sustain  the  average 
aviator's  gunnery  skills.  The  estimates  are  supported  by  the 
FY87  data  on  the  number  of  rounds  fired  (less  than  autho¬ 
rized)  and  the  percentage  of  units  that  did  not  meet  their 
training  readiness  standards. 

A  large  part  of  the  survey  information  about  ammunition 
requirements  and  training  utility  is  based  on  subjective 
opinions  rather  than  objective  data.  Although  the  survey 
data  provide  valuable  information,  further  research  that 
includes  longitudinal  surveys  and  empirical  experiments  is 
needed  to  determine  the  amount,  frequency,  and  type  of 
training  required  to  ensure  that  AA  and  NG  attack  helicopter 
units  are  capable  of  accomplishing  their  missions. 


Work  Projected 

All  the  scheduled  project  activities  have  been  completed 
and  no  further  work  is  anticipated  on  this  project. 
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UTILIZATION/EFFECTIVENESS  OF  FLIGHT  SIMULATORS 
FOR  AVIATION  UNIT  TRAINING 

Dr.  George  L.  Kaempf,  Project  Director 


Background 

Work  under  this  research  area  was  conducted  in  response 
to  two  taskings  by  the  Directorate  of  Training  and  Doctrine 
(DOTD )  to  the  Army  Research  Institute  Aviation  Research  and 
Development  Activity  (ARIARDA)  at  the  U.S.  Army  Aviation 
Center  (USAAVNC) ,  Fort  Rucker,  Alabama.  One  tasking  origi¬ 
nated  at  the  USAAVNC  and  the  other  originated  at  the 
Department  of  the  Army  (DA) .  The  taskings  have  been  dis¬ 
cussed  in  detail  previously  (see  Cross  &  Gainer,  1987; 
Kaempf,  1988;  Kaempf  &  Blackwell,  1988;  Kaempf,  Cross,  & 
Blackwell,  1989) .  Therefore,  only  a  brief  review  of  the 
taskings  is  presented  in  this  report. 


U.S.AAV.N.C  ..TasXiag 

The  Army  Audit  Agency  (AAA)  audited  the  Army's  Synthetic 
Flight  Training  System  (SFTS)  twice,  first  in  1981  and  again 
in  1984.  Reports  of  these  audits  (U.S.  Army  Audit  Agency, 
1982,  1985)  stated  that,  although  flight  simulators  had 
reduced  training  costs  and  improved  training  at  the  USAAVNC, 
the  Army  had  not  determined  the  effects  that  the  employment 
of  flight  simulators  may  have  on  training  in  operational 
aviation  units.  Both  reports  admonished  the  Army  for  the 
manner  in  which  operational  tests  had  been  conducted  on  the 
SFTS  and  concluded  that  the  Army  had  not  adequately  quanti¬ 
fied  the  return  on  its  investment  in  flight  simulators 
procured  for  aviation  unit  training. 

In  response  to  the  audits,  the  USAAVNC  tasked  DOTD  to 
initiate  research  that  addressed  the  issues  raised  by  AAA. 

In  1985,  DOTD  formally  tasked  ARIARDA  to  plan  and  conduct 
research  to  answer  such  questions  as: 

•  What  tasks  can  best  be  trained  in  flight  simulators? 

•  What  rate  of  practice  in  flight  simulators  best 
enables  aviators  to  maintain  proficiency? 

•  How  can  flight  simulators  be  used  to  sustain  profi¬ 
ciency  on  skills  that  are  not  routinely  practiced  in 
the  aircraft? 

•  What  effect  does  simulated  gunnery  training,  as  a 
substitute  for  live-fire  training,  have  on  aviator 
proficiency  and  unit  readiness? 
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As  a  result  of  the  tasking  from  USAAVNC,  ARIARDA  and 
Anacapa  initiated  three  research  projects  in  the  AH-1F  Flight 
and  Weapons  Simulator  (FWS) :  backward  transfer  and  skill 
acquisition  in  the  FWS,  transfer  of  training  in  the  FWS  for 
emergency  touchdown  maneuvers  (ETMs),  and  transfer  of 
training  in  the  FWS  for  gunnery  skills.  The  first  two 
projects  have  been  completed;  the  third  project  was  planned 
but  was  eventually  subsumed  under  the  work  conducted  for  the 
DA  tasking. 


DA  Tasking 

In  1986,  the  Office  of  the  Deputy  Chief  of  Staff  for 
Operations  at  DA  reviewed  the  issues  concerning  the  develop¬ 
ment  of  flight  simulation  training  programs  and  the  fielding 
of  flight  simulators.  DA  determined  that  training  effective¬ 
ness  analyses  should  be  conducted  for  each  of  rhe  Army's 
flight  simulation  systems.  DA  intended  the  analyses  to  serve 
as  the  basis  for  developing  effective  training  strategies  and 
programs . 

Research  plan.  In  June  1986,  DA  tasked  the  Training  and 
Doctrine  Command  (TRADOC)  to  develop  and  implement,  with  the 
assistance  of  ARIARDA,  post-fielding  training  ef fectiveness 
analyses  (TEAs)  of  the  Army's  visual  flight  simulator  sys¬ 
tems.  ARIARDA  and  Anacapa  developed  a  research  plan  (Army 
Research  Institute,  1986)  that  addressed  the  utilization  of 
flight  simulators  in  operational  environments.  In  December 

1986,  ARIARDA  submitted  the  research  plan  to  TRADOC .  In  June 

1987,  TRADOC  approved  the  TEA  research  plan  and  authorized 
ARIARDA  to  implement  the  research. 

Gunnery  training  focus.  Concurrent  with  the  TEA 
tasking,  the  Department  of  Gunnery  and  Flight  Systems  (DGFS) 
at  the  USAAVNC  revised  the  helicopter  gunnery  training  manual 
(FM  1-140).  The  revised  manual  (designated  TC  1-140)  speci¬ 
fies  the  gunnery  training  requirements  and  performance 
standards  for  aviation  units.  Before  issuing  the  revised 
manual,  however,  DGFS  requested  that  ARIARDA  incorporate  into 
the  TEAs  an  analysis  of  the  crew  gunnery  training  programs 
proposed  in  TC  1-140.  The  objectives  of  the  analysis  are 

(a)  to  determine  the  extent  to  which  simulators  can  be  used 
to  train  crew  gunnery  tasks  effectively  in  aviation  units  and 

(b)  to  determine  the  amount  of  resources  required  to  support 
crew  gunnery  training  programs. 

In  response  to  the  request  from  DGFS,  ARIARDA  agreed  to 
focus  the  TEA  program  on  the  effectiveness  of  flight  simula¬ 
tors  for  training  gunnery  tasks  in  attack  helicopter  aviation 
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units.  ARIARDA  and  Anacapa  planned  and  initiated  three 
research  projects  for  gunnery  tasks:  TEA  of  the  FWS  for 
Conducting  Gunnery  Training,  TEA  of  the  AH-64A  Combat  Mission 
Simulator  (CMS)  for  Conducting  Crew  Gunnery  Initial  Qualifi¬ 
cation  Training,  and  TEA  of  the  CMS  for  Conducting  Crew 
Gunnery  Proficiency  Sustainment  Training. 


Research  Area  Reorganization 

During  the  previous  contract  year,  the  research  area 
Utilization/Effectiveness  of  Flight  Simulators  for  Aviation 
Unit  Training  comprised  five  projects:  two  projects 
responded  to  the  USAAVNC  tasking  and  three  projects  responded 
to  the  DA  tasking.  During  the  current  contract  year,  Anacapa 
reorganized  the  five  projects  into  two  research  areas.  The 
three  projects  that  address  training  in  the  AH-1F  FWS  remain 
under  the  present  research  area.  The  two  projects  that 
address  training  in  the  AH-64A  CMS  were  transferred  to  a  new 
research  area  entitled  Training  Effectiveness  Analyses  of  the 
AH- 64 A  CMS.  The  AH-64  CMS  research  area  is  described  in  the 
next  section  of  this  summary  report. 


Problem 

The  Army  is  making  a  significant  investment  in  the 
development  and  acquisition  of  motion-based,  visual  flight 
simulators  for  its  rotary  wing  aircraft.  High  fidelity 
simulators  are  viewed  as  cost-effective  alternatives  to 
flight  training  in  the  aircraft.  Visual  flight  simulator 
systems  have  been  developed  for  the  AH- IF,  AH-64A,  CH-47D, 
and  UH-60  helicopters.  Deployment  to  aviation  units  of  the 
AH- IF  and  CH-47D  simulator  systems  has  been  completed;  the 
AH- 64 A  and  UH-60  simulator  systems  are  in  advanced  stages  of 
deployment  to  aviation  units. 

The  Army's  .primary  objective  for  flight  simulation  is  to 
provide  training  devices  in  which  operational  aviators  may 
sustain  their  flight  and  tactical  skills.  However,  little 
empirical  data  currently  exist  (a)  to  demonstrate  that  flight 
simulators  effectively  and  efficiently  provide  this  type  of 
training  and  (b)  to  guide  the  Army  in  developing  training 
programs  that  include  an  optimum  mix  of  training  conducted  in 
the  aircraft  and  flight  simulator.  Empirical  data  are  needed 
to  ensure  that  the  Army  receives  the  maximum  return  on  its 
investment  in  flight  simulators. 
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Research  Objectives 


The  overall  objective  of  this  research  area  is  to  eval¬ 
uate  the  effectiveness  of  the  U.S.  Army  flight  simulator 
systems  for  training  operational  aviators.  The  current 
research  projects  are  designed  to  meet  the  following 
objectives : 

•  develop  procedures  for  evaluating  flight  simulator 
training  effectiveness, 

•  identify  tasks  that  can  be  sustained  effectively  and 
efficiently  in  the  FWS, 

•  determine  the  effectiveness  of  the  FWS  and  the  CMS  for 
training  crew  gunnery  tasks  and  sustaining  crew 
gunnery  proficiency  in  operational  units, 

•  provide  the  data  needed  to  develop  training  programs 
that  comprise  an  optimal  mix  of  flight  simulator  and 
aircraft  flight  training  in  aviation  units,  and 

•  identify  training  techniques  and  strategies  that  will 
enable  aviators  to  sustain  their  flight  and  tactical 
skills  through  training  in  flight  simulators. 


Research  Approach 

Backward  Transfer  and  Skill  Acquisition  in  the  FWS 

Two  experiments  were  planned  for  this  project.  The 
purposes  of  the  first  experiment  were  to  determine  if  the 
AH- IF  aircraft  and  the  FWS  are  interchangeable  training 
devices  and  to  test  the  utility  of  the  backward  transfer 
paradigm.  In  this  experiment,  AH- IF  instructor  pilots  at  the 
USAAVNC,  who  were  proficient  in  the  AH-1F  but  unfamiliar  with 
the  FWS,  were  evaluated  in  the  aircraft  and  the  FWS  on  their 
performance  of  eight  maneuvers,  including  five  emergency 
touchdown  maneuvers  (ETMs) .  The  backward  transfer  of  skills 
was  evaluated  by  comparing  pilot  performance  in  the  aircraft 
to  pilot  performance  m  the  FWS. 

The  purpose  of  the  second  experiment  was  to  estimate  how 
many  practice  trials  were  required  to  reach  a  satisfactory 
level  of  proficiency  on  each  maneuver.  In  this  experiment, 
four  different  groups  of  aviators  from  aviation  units  each 
received  simulator  training  on  a  different  set  of  five  maneu¬ 
vers.  The  subjects  received  a  maximum  of  ten  practice  trials 
on  each  maneuver. 
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Transfer  of  Training  in  the  FWS  for  ETMs 


This  project  was  designed  to  evaluate  the  effectiveness 
of  the  FWS  for  training  five  ETMs  that  aviators  assigned  to 
aviation  units  normally  are  prohibited  from  practicing  in  the 
aircraft.  The  Army  granted  an  exception  to  the  prohibition 
for  this  experiment.  Aviation  unit  aviators  were  given 
checkrides  in  the  AH- IF  aircraft  and  in  the  FWS;  they  were 
then  assigned  to  a  control  group  or  an  experimental  group. 

The  control  group  subjects  were  trained  to  proficiency  on  the 
ETMs  in  the  aircraft.  The  experimental  group  subjects  were 
trained  to  proficiency  on  the  ETMs  in  the  FWS  and  then 
trained  to  proficiency  in  the  aircraft.  The  effectiveness  of 
the  simulator  training  was  tested  by  comparing  the  two  groups 
on  their  performance  during  training  in  the  aircraft. 


Training  Effectiveness  Analysis  of  the  FWS  for  Conducting 
Gunnery  Training 

This  project  is  designed  to  evaluate  the  effectiveness 
of  the  FWS  for  sustaining  crew  gunnery  skills  in  aviation 
units.  Fifty  current  AH-1F  aviators  in  U.S.  Army,  Europe 
(USAREUR)  aviation  units  will  be  assigned  to  one  of  three 
groups:  one  control  and  two  experimental.  Each  suw sect's 

initial  proficiency  on  crew  gunnery  tasks  will  be  evaluated 
during  a  pretest  live-fire  gunnery  exercise. 

Following  the  pretest  live-fire  evaluation,  the  subjects 
will  enter  a  13-month  gunnery  training  program.  In  addition 
to  the  normal  unit  training,  the  Experimental  Group  1  avia¬ 
tors  will  receive  gunnery  training  in  the  FWS  every  month  and 
the  Experimental  Group  2  aviators  will  receive  gunnery 
training  in  the  FWS  every  3  months.  However,  aviators  in 
both  experimental  groups  will  be  restricted  on  the  gunnery 
tasks  that  they  can  practice  in  the  aircraft.  The  Control 
Group  aviators  will  receive  the  unit's  normal  training  and 
will  not  be  restricted  on  gunnery  practice  in  the  aircraft. 
The  effectiveness  of  the  FWS  for  sustaining  gunnery  skills 
will  be  tested  by  comparing  the  performance  of  the  three 
groups  during  a  posttest  live-fire  exercise. 


Work  Completed 

Backward  Transfer  and  Skill  Acquisition  in  the  AH1FWS 

All  data  collection  and  analyses  were  completed  during  a 
previous  contract  year,  and  a  report  was  written  to  present 
the  findings  of  the  experiment.  During  the  current  contract 
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year,  Anacapa  edited  the  report  and  submitted  it  in  final 
format  (Kaempf  et  al . ,  1569)  to  ARIARDA. 


Transfer  of  Training  in  the  AH1FWS  for  ETMs 

All  data  collection  and  analyses  were  completed  during  a 
previous  contract  year  and  a  report  (Kaempf  &  Blackwell, 

1988)  was  written  to  present  the  findings  of  the  experiment. 
During  the  current  contract  year,  Anacapa  edited  the  report 
and  submitted  it  to  ARIARDA. 


Training  Effectiveness  Analysis  of  the  AH1FWS  for  Conducting 
Gunnery  Training 

In  October  1988,  USAREUR  officials  authorized  ARIARDA  to 
conduct  the  TEA  of  the  FWS  and  tasked  V  and  VII  Corps  to 
provide  resources  to  support  the  experiment.  Three  aviation 
units  identified  25,  13,  and  12  AH-1F  aviators,  respectively, 
to  serve  as  subjects.  Because  of  logistical  and  scheduling 
considerations,  each  of  the  aviation  units  entered  the 
experiment  as  its  training  schedule  permitted  and  proceeded 
independently  of  the  other  two  units. 

An  AH-1F  Aviator  Questionnaire  was  developed  and  admin¬ 
istered  to  AH- IF  aviators  in  USAREUR  units.  The  AK-1F 
Aviator  Questionnaire  contains  44  items  requesting  informa¬ 
tion  about  personal  history,  flight  experience,  current  duty 
assignment,  experience  with  AH-1F  weapon  systems,  and  opin¬ 
ions  about  flight  and  gunnery  training.  The  objectives  of 
the  questionnaire  are  to  describe  (a)  the  subjects  partici¬ 
pating  in  the  TEA  and  (b)  the  population  of  AH-1F  aviators 
from  which  the  subjects  were  selected.  Approximately  200 
AH- IF  aviators  responded  to  the  questionnaire,  including  the 
50  TEA  subjects. 

Two  other  data  collection  instruments  were  developed  for 
use  in  the  TEA:  the  Diagnostic  Gunnery  Skills  Test  and  the 
Postflight  Debriefing  Form.  The  Diagnostic  Gunnery  Skills 
Test  comprises  32  items  designed  to  test  the  subjects'  knowl¬ 
edge  of  the  AH- IF  weapon  systems  and  gunnery  techniques.  All 
subjects  completed  the  test  prior  to  their  pretest  live-fire 
exercise.  The  Postflight  Debriefing  Form  comprises  16  items 
that  document  the  training  that  the  subjects  receive  in  the 
aircraft  in  conjunction  with  their  training  in  the  FWS. 
Gunnery  and  tactical  tasks  are  emphasized  on  the  form.  All 
subjects  were  required  to  complete  a  debriefing  form  immedi¬ 
ately  after  each  flight  in  the  aircraft. 
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The  three  participating  units  conducted  their  pretest 
live-fire  exercises  in  January,  April,  and  May  1989,  respec¬ 
tively.  During  the  pretest  live-fire  exercises,  each  aviator 
was  required  to  complete  Table  VII,  Crew  Training,  and  Table 
VIII,  Crew  Qualification,  of  TC  1-140  (U.S.  Army  Aviation 
Center,  1988) .  Each  table  comprises  ten  engagements  accom¬ 
plished  from  ten  different  firing  positions.  Data  collectors 
recorded  measures  of  target  effect,  total  engagement  time, 
and  exposure  time  for  each  engagement. 

Following  the  pretest  live-fire  exercises,  Experimental 
Groups  1  and  2  began  the  TEA  training  program.  The  TEA 
training  program  was  designed  to  sustain  proficiency  on 
specific  gunnery  tasks  within  the  context  of  mission 
scenarios.  During  each  FWS  training  session,  the  subjects 
planned  and  executed  an  attack  mission  prepared  by  Anacapa 
personnel  and  AH- IF  standardization  pilots.  The  attack 
mission  differed  for  each  FWS  training  session.  During  each 
training  session  in  the  FWS,  a  data  collector  recorded 
several  performance  measures,  including  total  engagement 
time,  target  effect,  flight  time  in  each  crew  station,  and 
number  of  rounds  expended. 

In  August  1989,  ARIARDA  directed  Anacapa  to  terminate 
work  on  the  FWS  TEA  no  later  than  30  August  1989.  On  28-29 
August,  the  Anacapa  Sciences  Project  Director  briefed  ARIARDA 
personnel  about  the  project’s  status  and  discontinued  work  on 
the  project  . 


Work  Projected 

Backward  Transfer  and  Skill  Acquisition  in  the  AH1FWS 

No  further  work  is  planned  for  this  project.  Submission 
of  the  report  to  ARIARDA  completed  all  activities  for  this 
project . 


Transfer  of  Training  in  the  AH1FWS  ..for  ..ETMs 

No  further  work  is  planned  for  this  project.  Submission 
of  the  report  to  ARIARDA  completed  all  activities  for  this 
project . 
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No  further  work  is  planned  for  this  project  by  Anacapa 
Sciences  personnel.  ARIARDA  personnel  have  assumed  responsi¬ 
bility  for  continuing  the  data  collection  phase  of  the 
project . 
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TRAINING  EFFECTIVENESS  ANALYSIS  OF  THE 
AH-64A  COMBAT  MISSION  SIMULATOR 

Dr.  Richard  Weeter,  Project  Director 


Background 

Five  projects  evaluating  the  effectiveness  of  flight 
simulators  were  originally  planned  under  the  research  area 
entitled  Utilization/Effectiveness  of  Flight  Simulators  for 
Aviation  Unit  Training  (see  pp. 71-79).  In  April  1989, 

Anacapa  reorganized  these  projects .  The  three  research 
projects  already  in  progress  on  AH-1  Flight  and  Weapons 
Simulator  (FWS)  issues  remain  under  the  original  research 
area;  the  two  projects  planned  to  address  AH-64A  Combat 
Mission  Simulator  (CMS)  issues  were  transferred  into  the 
present  research  area. 

The  two  CMS  projects  were  developed  in  response  to  a 
Department  of  the  Army  (DA)  tasking  to  the  Army  Research 
Institute  Aviation  Research  and  Development  Activity 
( ARIARDA)  in  1986.  Following  a  request  from  the  Department 
of  Gunnery  and  Flight  Systems  (DGFS)  in  1988,  the  projects 
were  modified  to  focus  on  helicopter  gunnery  issues.  The  DA 
tasking  and  the  DGFS  request  are  discussed  in  detail  in  Cross 
and  Gainer  (1987)  and  in  Kaempf  (1988).  Therefore,  only 
summaries  of  the  DA  tasking  and  the  DGFS  request  are 
presented  below. 


DA  Tasking 

In  1986,  DA  tasked  ARIARDA,  through  the  Training  and 
Doctrine  Command  (TRADOC) ,  to  plan  and  initiate  post-fielding 
training  effectiveness  analyses  (TEAs)  of  each  of  the  Army's 
flight  simulator  systems.  The  TEAs  were  intended  to  investi¬ 
gate  the  utilization  and  training  effectiveness  of  Army 
flight  simulator  systems  in  operational  field  units  and  to 
provide  a  basis  for  developing  effective  training  strategies. 
In  response  to  the  tasking,  ARIARDA  and  Anacapa  developed  a 
research  plan  comprising  a  series  of  related  research 
projects  (Army  Research  Institute,  1986).  Each  project  was 
designed  to  investigate  the  effectiveness  of  a  flight 
simulator  system  for  training  a  specific  set  of  tasks  (e.g., 
weapons  tasks,  contact  flight  tasks,  and  emergency  flight 
tasks)  in  an  operational  environment. 
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DGFS  Request 


Concurrent  with  the  DA  tasking,  DGFS  proposed  revisions 
to  the  helicopter  gunnery  training  manual,  FM  1-140.  FM 
1-140  defines  the  training  requirements  and  performance 
standards  for  the  Army's  aerial  gunnery  training  program. 

The  revised  manual  (TC  1-140)  contains  significant  changes  to 
the  crew  gunnery  training  requirements  for  AH-64  aircraft . 

In  TC  1-140,  DGFS  proposes  to  conduct  all  AH-64A  crew  gunnery 
training  and  qualification  in  the  CMS.  That  is,  no  live 
rounds  would  be  provided  for  crew  training  and  qualification. 
Live  rounds  would  be  expended  only  for  training  attack  heli¬ 
copter  teams  and  for  conducting  combined  arms  live-fire  and 
joint  air  attack  team  exercises.  Before  issuing  the  revised 
manual,  however,  DGFS  requested  that  ARIARDA  conduct  research 
to  evaluate  the  effectiveness  of  the  FWS  and  the  CMS  for  crew 
gunnery  training  and  qualification. 

ARIARDA  agreed  to  focus  the  TEAS  on  the  issues  raised  by 
the  DGFS.  Two  projects  were  designed  to  investigate  gunnery 
training  in  the  CMS:  TEA  of  the  CMS  for  Conducting  Crew 
Gunnery  Initial  Qualification  Training  and  TEA  of  the  CMS  for 
Conducting  Crew  Gunnery  Proficiency  Sustainment  Training. 


Problem 

The  Army  uses  high  fidelity  flight  simulators  to  augment 
and,  in  some  cases,  replace  the  training  that  aviators 
receive  in  aircraft.  The  Synthetic  Flight  Training  System 
( SFTS )  program  is  viewed  as  a  cost-effective  means  of 
acquiring  flight  skills.  In  TC  1-140,  DGFS  proposes  that 
flight  simulators  may  also  be  an  effective  alternative  to 
live-fire  training  in  the  aircraft  for  both  the  acquisition 
and  sustainment  of  crew  gunnery  skills. 

However,  there  is  little  empirical  data  to  demonstrate 
the  effectiveness  of  flight  simulators  in  augmenting 
training.  Empirical  data  are  required  to  demonstrate  that 
flight  simulators  can  effectively  train  and  sustain  flight  or 
tactical  skills.  Furthermore,  empirical  data  are  required  to 
provide  guidelines  for  establishing  an  optimal  combination  of 
aircraft  and  simulator  training  for  specific  tasks  such  as 
gunnery . 


Research  Objectives 

The  research  in  this  area  is  designed  to  meet  three 
major  objectives: 
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•  determine  the  effectiveness  of  the  CMS  for  the 
acquisition  of  crew  gunnery  skills, 

•  determine  the  effectiveness  of  the  CMS  for  the 
sustainment  of  crew  gunnery  skills,  and 

•  provide  data  to  help  establish  an  optimum  combination 
of  aircraft  and  flight  simulator  training  for  the 
acquisition  and  sustainment  of  crew  gunnery  skills. 


Research  Approach 
Initial  Qualification  Project 

The  initial  qualification  project  is  designed  to  eval¬ 
uate  the  training  effectiveness  of  the  CMS  for  the  acquisi¬ 
tion  of  crew  gunnery  skills.  Novice  crews  from  the  AH-64A 
Apache  Training  Brigade  (ATB)  will  be  assigned  to  either  a 
control  or  an  experimental  group.  The  control  group  will 
receive  the  normal  ATB  flight  and  gunnery  training  in  the  AH- 
64 A .  The  experimental  group  will  conduct  their  flight 
training  in  the  AH-64A  and  their  gunnery  training  in  the  CMS. 
The  effectiveness  of  the  two  training  programs  will  be  tested 
by  comparing  the  crews'  performance  during  a  live-fire 
gunnery  exercise  following  an  8-week  training  period. 


Proficiency  Sustainment  Project 

The  proficiency  sustainment  project  is  designed  to  eval¬ 
uate  the  effectiveness  of  the  CMS  for  sustaining  gunnery 
skills  for  1  year  without  live-fire  practice.  Fully  quali¬ 
fied  AH-64A  crews  from  the  6th  Calvary  Brigade  -  Air  Combat 
(CBAC )  will  be  assigned  to  either  a  control  group  or  one  of 
two  experimental  groups.  The  crews'  baseline  gunnery  skills 
will  be  measured  during  a  pretest  live-fire  gunnery  exercise. 
All  CMS  and  aircraft  training  received  by  each  group  will  be 
controlled  for  1  year  after  the  pretest  live-fire  gunnery 
exercise . 

The  control  group  will  receive  the  normal  program  of 
instruction  in  both  the  CMS  and  the  aircraft  throughout  the 
i-year  test  period.  Experimental  Group  One  will  also  receive 
the  normal  program  of  instruction  in  the  CMS,  but  will 
receive  additional  gunnery-specific  training  in  the  aircraft 
(dry-fire  on  the  gunnery  range)  during  the  test  period. 
Experimental  Group  Two  will  receive  the  normal  training  in 
the  aircraft  and  additional  gunnery-specific  training  in  the 
CMS  during  the  test  period.  The  amount  of  gunnery  training 
received  in  addition  to  the  normal  training  will  be  equated 
for  Experimental  Groups  One  and  Two. 
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After  1  year,  the  crews'  gunnery  skills  will  be  eval¬ 
uated  during  a  posttest  live-fire  gunnery  exercise.  The 
ef f ect iveness  of  the  CMS  will  be  evaluated  by  (a)  comparing 
performance  between  the  pretest  exercise  and  the  posttest 
exercise  and  (b)  comparing  the  performance  of  the  three 
groups  during  the  posttest  exercise. 


Work  Completed 

Initial  Qualification  Project 

Coordination  problems  with  the  ATB  have  delayed  the 
initiation  of  this  project.  During  the  current  contract 
year,  no  research  was  conducted  on  initial  gunnery  qualifi¬ 
cation  in  the  CMS. 


Proficiency  Sustainment  Project 

In  April  1989,  baseline  live-fire  exercise  data  were 
collected  at  the  Dalton-Henson  Multi-Purpose  Range  Complex  at 
Fort  Hood,  Texas.  At  that  time,  the  6th  CBAC  could  assign 
only  15  crews  to  the  project  because  of  anticipated  personnel 
turnover.  Consequently,  baseline  gunnery  evaluations  were 
scheduled  for  additional  6th  CBAC  crews  during  the  next  live- 
fire  exercise  in  August. 

In  August  1989,  baseline  live-fire  data  were  collected 
from  an  additional  12  crews,  but  4  of  the  crews  tested  in 
April  were  no  longer  able  to  participate  in  the  research.  At 
the  close  of  the  contract  year,  there  were  9  crews  in  the 
control  group,  8  crews  in  Experimental  Group  One,  and  6  crews 
in  Experimental  Group  Two.  During  September,  data  were 
collected  during  the  first  of  10  scheduled  gunnery-specific 
training  sessions  in  the  CMS. 


Work  Projected 

Initial  Qualification  Project 

During  the  next  contract  year,  project  personnel  will 
meet  with  the  ATB  to  determine  if  ATB  will  support  the 
initial  qualification  research. 
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Proficiency  Sustainment  Project 


During  the  next  contract  year,  all  experimental  training 
sessions  and  data  collection  activities  for  this  project  will 
be  completed.  During  the  training  period,  project  personnel 
will  monitor  (a)  the  type  and  amount  of  aircraft  and  simu¬ 
lator  training  all  crews  receive,  (b)  the  number  of  dry-fire 
iterations  that  the  crews  in  Experimental  Group  One  receive, 
and  (c)  the  remaining  nine  experimental  gunnery-specific 
training  sessions  in  the  CMS. 

Data  collection  during  the  posttest  live-fire  exercise 
for  aviators  who  were  pretested  in  April  1989  is  scheduled 
for  March  1990.  The  posttest  live-fire  exercise  for  aviators 
who  began  the  project  in  August  1989  is  scheduled  for  August 
1990.  Following  the  August  1990  live-fire  exercise,  the  data 
will  be  analyzed  and  a  draft  report  will  be  prepared.  The 
draft  report  will  be  edited  and  submitted  to  ARIARDA  early  in 
the  following  contract  year. 
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EFFECTIVENESS  OF  AIRNET  IN  TRAINING  COLLECTIVE  TASKS 
Dr.  Beth  W.  Thomas,  Project  Director 


Background 

Recent  assessments  by  the  Directorate  of  Combat  Develop¬ 
ments  (1982,  1983,  1986)  at  the  U.S.  Army  Aviation  Center 
(USAAVNC) ,  Fort  Rucker,  Alabama,  identified  several  aviation 
training  deficiencies,  primarily  in  training  collective  or 
mission  tasks.  The  training  deficiencies  identified  in  the 
Battlefield  Development  Plan  (1986)  include  air-to-air  opera¬ 
tions,  anti-armor  operations,  sustained  aviation  operations, 
air  assault  operations,  suppression  of  enemy  air  defense, 
special  operations  missions,  aerial  reconnaissance,  combat 
maneuvers,  search  and  rescue  operations,  target  acquisition 
and  handover,  and  aircraft  survivability.  Currently,  the 
deficient  tasks  are  not  trained  effectively  in  helicopters 
because  of  several  factors;  these  factors  include  a  lack  of 
threat  interaction,  a  lack  of  combined  arms/joint  training,  a 
lack  of  resources  (e.g.,  flight  hours,  missiles),  and  safety 
concerns . 

The  current  constraints  on  training  in  the  aircraft  are 
likely  to  inhibit  Army  aviation  training  in  the  future. 
Therefore,  there  is  a  need  to  evaluate  simulation  as  a 
supplement  to  institutional  and  unit  aviation  training.  The 
Directorate  of  Training  and  Doctrine  (DOTD)  began  work  on  the 
development  and  acquisition  of  an  Aviation  Combined  Arms  Team 
Trainer  in  1983,  but  the  cost  of  building  a  prototype  device 
became  so  prohibitive  that  the  program  was  discontinued  in 
1985. 

DOTD  then  began  to  investigate  alternatives  to  develop¬ 
ing  a  completely  new  aviation  simulator.  At  that  time,  the 
Defense  Advanced  Research  Projects  Agency  (DARPA)  was 
sponsoring  a  research  and  development  program  on  low  cost, 
low  fidelity  simulation  networking  (SIMNET) .  In  SIMNET, 
several  simulators  at  the  same  location  are  networked  to 
allow  individual  crews  to  interact  with  one  another  on  the 
same  data  base.  SIMNET  simulates  only  ground  force  vehicles, 
the  Ml  tank  and  the  M2/3  Bradley  fighting  vehicle.  The  cost 
of  the  networked  system  .i  s  kept  low  by  including  only  the 
systems  in  the  vehicle  that  are  considered  mission  essential. 

In  1987,  USAAVNC  established  a  memorandum  of  under¬ 
standing  with  DARPA  for  the  development  of  an  aviation 
counterpart  to  SIMNET  to  be  called  AIRNET.  The  AIRNET  device 
is  to  serve  as  the  proof-of-concept  of  networking  for  avia¬ 
tion  purposes.  The  device  is  to  be  developed  in  three 
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phases:  Fully  Reconf igurable  Experimental  Device  (FRED), 

Generic,  and  60%  Solution.  The  operational  device  will  be 
called  the  Aviation  Combined  Arms  Tactical  Trainer  (AVCATT) . 

On  16  June  1988,  DOTD  requested  that  the  Army  Research 
Institute  Aviation  Research  and  Development  Activity 
(ARIARDA)  evaluate  the  ability  of  the  AIRNET  device  to 
support  the  training  of  Army  Training  and  Evaluation  Program/ 
Mission  Training  Plan  (ARTEP/MTP)  tasks  (ARTEP,  1988)  .  Of 
particular  importance  were  the  tasks  that  would  potentially 
reduce  the  deficiencies  identified  in  the  Battlefield 
Development  Plan.  In  addition,  ARIARDA  was  asked  to  provide 
information  about  changes  that  are  required  to  accommodate 
the  mission  tasks  that  cannot  be  performed  in  the  device. 
Finally,  DOTD  requested  that  evaluations  be  conducted  at  each 
phase  of  development  to  determine  how  much  fidelity  is 
necessary  and  sufficient  to  support  mission  training.  In 
September  1988,  Anacapa  Sciences  began  developing  a  research 
plan  to  evaluate  the  AIRNET  device. 


Research  Objectives 

The  general  objectives  of  this  research  are  to  evaluate 
the  effectiveness  of  the  AIRNET  device  for  training  collec¬ 
tive  ARTEP/MTP  tasks  to  reduce  the  deficiencies  identified  in 
the  Battlefield  Development  Plan  and  to  provide  input  to  the 
follow-on  device,  AVCATT.  Therefore,  the  specific  objectives 
of  the  research  project  are  to  identify: 

•  the  ARTEP/MTP  tasks  for  the  Attack  Helicopter  Company 
and  the  Air  Cavalry/Reconnaissance  Troop  that  can  be 
performed  in  the  AIRNET  device, 

•  the  changes  in  the  device  that  are  required  to  accom¬ 
modate  the  ARTEP/MTP  tasks  that  cannot  be  performed  in 
AIRNET, 

•  the  training  deficiencies  listed  in  the  Battlefield 
Development  Plan  (1986)  that  may  be  addressed  by  the 
AIRNET  device, and 

•  the  mission  casks  in  the  Aeroscout  Observer  course 
that  can  be  trained  effectively  in  the  AIRNET  device. 


Research  Approach 

Separate  evaluations  will  be  conducted  for  each  phase  in 
the  development  of  the  AIRNET  device.  In  the  first  evalua¬ 
tion,  aviation  test  pilots  and  instructor  pilots  (IPs)  will 
"fly"  in  the  FRED  to  determine  its  capability  for  performing 
basic  flight  maneuvers.  The  comments  and  recommendations  of 
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the  pilots  and  IPs  will  be  used  to  upgrade  the  FRED  aeromodel 
for  use  in  the  Generic  device . 


The  evaluation  of  the  Generic  device  will  be  conducted 
in  three  steps.  First,  test  pilots  and  IPs  will  fly  in  the 
Generic  device  to  evaluate  its  capability  for  performing 
basic  flight  maneuvers.  After  any  necessary  corrections  are 
made  to  the  aeromodel,  a  formal  evaluation  will  be  conducted 
to  determine  if  the  Aircrew  Training  Manual  (ATM)  and  ARTEP/ 
MTP  tasks  can  be  performed  in  the  Generic  device.  During 
this  evaluation,  aviation  unit  aviators  will  be  trained  to 
operate  the  Generic  device;  they  will  then  be  required  to 
conduct  combined  scout/attack  helicopter  missions  against 
enemy  threat  vehicles. 

At  least  four  types  of  performance  data  will  be  col¬ 
lected  and  evaluated:  automated  system  performance  measures, 
ratings  of  task  performance,  questionnaire  data  about  mission 
task  performance,  and  reliability,  availability,  and  main¬ 
tainability  (RAM)  data.  The  results  of  the  formal  evaluation 
with  operational  pilots  will  be  used  to  modify  the  Generic 
device . 

Finally,  a  transf er-of-training  investigation  will  be 
conducted  with  Aeroscout  Observer  (AO)  trainees.  Some  AO 
trainees  will  receive  mission  training  in  the  aircraft  and 
other  AO  trainees  will  receive  mission  training  in  the 
Generic  AIRNET  device.  Differences  in  the  effectiveness  of 
the  training  groups  will  be  determined  by  comparing  their 
performance  of  mission  tasks  in  the  OH-58  aircraft .  The 
results  of  this  evaluation  will  be  used  to  determine  the 
potential  utility  of  AIRNET  for  enhancing  institutional 
training . 

The  evaluation  of  the  60%  Solution  AIRNET  device  will  be 
similar  to  the  Generic  AIRNET  evaluation.  First,  test  pilots 
and  IPs  will  fly  in  the  device  to  evaluate  the  capability  of 
the  aeromodel  for  performing  the  basic  flight  maneuvers.  If 
corrections  are  required,  they  will  be  made  prior  to  the 
conduct  of  the  ARTEP/MTP  evaluation.  During  the  formal  eval¬ 
uation,  operational  unit  aviators  will  be  trained  to  operate 
the  device  and  evaluated  while  they  conduct  team  training 
exercises  against  an  automated  enemy  threat.  At  least  four 
types  of  performance  data  will  be  collected  and  evaluated: 
automated  system  performance  measure,  subjective  ratings  of 
task  performance,  questionnaire  data  about  mission  task 
performance,  and  RAM  data.  If  required,  additional 
performance  measures  will  be  developed  specifically  for  the 
60%  Solution  evaluation.  The  results  of  the  evaluation  of 
the  60%  Solution  AIRNET  device  will  be  used  to  produce  the 
AVCATT  prototype  device . 
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Work  Completed 


The  evaluation  of  the  FRED  was  conducted  during  November 
and  December,  1988.  Four  unit  IPs  and  three  Directorate  of 
Evaluation  and  Standardization  IPs  served  as  subject  matter 
experts  in  this  evaluation.  The  flight  characteristics  of 
the  device  were  so  unlike  any  helicopter  that  all  individual 
flight  tasks  were  virtually  impossible  to  perform  within 
acceptable  standards.  To  correct  these  obvious  problems,  two 
pilots  from  the  Aviation  Development  Test  Activity  (ADTA) 
were  requested  to  evaluate  the  aeromodel  of  the  AIRNET 
device.  The  ADTA  test  pilots  identified  the  minimum  changes 
in  the  flight  and  handling  characteristics  needed  to  allow 
flight.  The  comments  from  the  IPs  and  ADTA  test  pilots  were 
submitted  to  DOTD .  The  recommended  changes  and  other  planned 
additions  were  made  in  developing  the  Generic  AIRNET  device. 

Prior  to  the  ARIARDA  evaluation  on  the  Generic  AIRNET 
device,  the  ADTA  test  pilots  were  asked  to  evaluate  the  new 
flight  aeromodel.  The  pilots  provided  a  report  suggesting 
further  changes  that  were  necessary  to  decrease  the  workload 
required  to  fly  the  device.  This  informaticn  was  forwarded 
to  DOTD  and  the  changes  to  the  aeromodel  were  made  within  the 
next  month. 

The  Directorate  of  Combat  Developments  developed 
scenarios  that  incorporated  the  ARTEP/MTP  tasks  in  a  logi¬ 
cally,  tactically,  and  doctrinally  correct  manner.  The 
scenarios  were  adapted  to  the  constraints  of  the  AIRNET 
terrain.  With  the  exception  of  ground  tasks  (e.g.,  preflight 
checks)  and  the  flight  tasks  concerning  Nuclear,  Biological, 
and  Chemical  (NBC)  warfare,  all  of  the  ARTEP/MTP  mission 
tasks  for  the  Attack  Helicopter  Company  and  Air  Cavalry 
Reconnaissance  Troop  were  evaluated  during  the  formal  evalua¬ 
tion.  Many  ground  tasks  were  performed  in  preparation  for 
the  mission  (e.g.,  company  commander  briefs  the  mission),  but 
were  not  emphasized  in  the  investigation. 

Rating  scales  were  developed  for  each  mission  task  for 
both  scout  and  attack  crews.  The  scales  were  evaluated  by 
subject  matter  experts  and  modifications  were  made  where 
necessary  to  clarify  the  criteria  for  successful  completion 
of  the  mission  tasks.  Raters  were  then  trained  to  use  the 
scales.  Interobserver  reliability  was  1.0  for  each  mission 
task  for  both  scout  and  attack  rating  teams. 

Mission  task  and  technical  performance  questionnaires 
were  developed  to  be  administered  to  the  individuals  partici¬ 
pating  in  the  evaluation.  Short  versions  of  the  question¬ 
naire  (Form  A)  were  administered  prior  to  and  following  the 
collective  evaluation  of  the  Generic  device.  The  longer 
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questionnaire  (Form  B)  included  questions  on  simulator 
sickness,  negative  habit  transfer,  and  relative  training 
value  of  AIRNET;  Form  B  was  administered  on  the  final  day  of 
the  evaluation. 

From  21  July  to  7  September  1989,  one  operational  attack 
helicopter  unit  participated  in  the  formal  evaluation  of  the 
Generic  AIRNET  device.  The  unit  was  divided  into  two  teams: 
one  team  was  composed  of  two  scout  and  two  attack  heli¬ 
copters;  the  other  team  was  composed  of  one  scout  and  two 
attack  helicopters.  The  investigation  included  training  on 
the  operation  of  the  device,  a  pretest  of  performance  on  the 
individual  ATM  tasks,  an  evaluation  of  performance  on  collec¬ 
tive  mission  tasks,  and  a  posttest  of  performance  on  the 
individual  ATM  tasks.  In  addition,  the  reliability,  avail¬ 
ability,  and  maintainability  of  the  system  was  evaluated  by  a 
RAM  engineer  from  DOTD . 

A  preliminary  summary  report  of  the  evaluation  was 
requested  by  DOTD  so  that  recommended  changes  could  be  inte¬ 
grated  into  the  design  of  the  60%  Solution  AIRNET  device.  A 
draft  preliminary  report  was  submitted  to  ARIARDA  in  Septem¬ 
ber  1989.  Following  a  review  by  ARIARDA,  the  draft  prelimi¬ 
nary  summary  report  was  revised  and  submitted  in  final  form 
on  3  October  1989  (Thomas,  1989) .  The  report  documents 
several  deficiencies  in  the  Generic  device.  In  particular, 
corrections  to  the  aeromodel  are  still  required  to  enable  the 
aviators  to  perform  the  mission  tasks.  The  reliability  of 
the  device  needs  to  be  improved;  the  maintenance  required  to 
operate  the  devices  is  excessive.  The  results  indicate  that 
networked  simulators  are  potentially  useful  for  conducting 
combined  arms  and  joint  training;  however,  the  present 
version  of  the  Generic  AIRNET  device  does  not  meet  the  AVCATT 
objectives.  In  addition,  further  research  is  required  to 
determine  the  training  effectiveness  of  the  AIRNET  device  for 
the  ARTEP/MTP  tasks. 


Work  Projected 

The  final  report  on  the  evaluation  of  the  Generic  AIRNET 
device  will  be  prepared  and  submitted  when  all  of  the  analy¬ 
ses  have  been  completed.  The  final  report  will  provide  more 
detailed  information  about  the  results  presented  in  the 
preliminary  summary  report  and  will  include  additional  infor¬ 
mation  that  was  not  available  at  the  time  the  preliminary 
report  was  written. 

The  training  effectiveness  evaluation  of  AIRNET  for  use 
in  the  AO  course  is  scheduled  for  November  1989.  Mission 
tasks  trained  in  the  device  will  be  evaluated  against 
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performance  in  the  aircraft.  The  mission  tasks  will  be  as 
follows:  call  for  and  adjust  fires,  perform  actions  on 
contact,  provide  spot  reports,  and  acquire  and  handover 
targets.  IPs  for  the  AO  course  will  conduct  the  training  in 
the  AIRNET  device  and  the  evaluations  in  the  aircraft. 

Anacapa  and  ARIARDA  will  also  conduct  an  ARTEP/MTP 
evaluation  on  the  AIRNET  60%  Solution  device  when  it  is 
installed  at  Fort  Rucker. 
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DEVELOPMENT  OF  THE  AH-64A  DISPLAY  SYMBOLOGY  TRAINING  MODULE 
Dr.  John  W.  Ruffner,  Project  Director 


Background 

The  AH-64A  attack  helicopter  is  a  two- crewmember  air¬ 
craft  designed  to  fly  nap-of-the-earth  missions  to  detect, 
engage,  and  destroy  enemy  armor  during  day  or  night  and  in 
all  weather  conditions.  To  provide  this  capability,  the 
AH-64A  is  equipped  with  complex  flight  and  weapons  delivery 
systems.  The  successful  operation  of  these  systems  requires 
that  the  pilot  and  copilot/gunner  (CPG)  be  able  to  identify 
and  interpret  both  visual  imagery  and  symbolic  information 
presented  on  visual  displays. 

The  AH-64A  visual  display  systems  that  provide  informa¬ 
tion  to  the  pilot  and  the  CPG  are  the  Pilot  Night  Vision 
System  (PNVS)  and  the  Target  Acquisition  and  Detection  System 
(TADS) .  The  PNVS  provides  forward-looking  infrared  (FLIR) 
imagery  that  enables  the  pilot  to  fly  the  aircraft  at  night 
and  during  degraded  visibility  conditions.  The  TADS  is  used 
by  the  CPG  for  target  search,  detection,  recognition,  and 
designation.  The  TADS  uses  information  from  three  sensors: 
the  FLIR  system,  the  day  television  viewing  system,  and  the 
direct  view  optics  system.  These  three  sensors  provide  the 
CPG  with  visual  information  to  detect  and  engage  targets  at 
standoff  ranges  during  day  or  night  operations  and  in  adverse 
weather  conditions.  The  Fire  Control  Symbol  Generator 
superimposes  flight  and  weapons  symbology  on  the  imagery 
displayed  by  the  PNVS  and  the  TADS. 

The  visual  imagery  and  symbology  from  the  PNVS  and  the 
TADS  can  be  presented  to  the  pilot  on  a  4.0  by  5.0  inch 
panel-mounted  display  or  to  the  CPG  on  a  2.25  by  3.25  inch 
panel-mounted  display.  In  addition,  the  imagery  and 
symbology  can  be  presented  to  either  crewmember  through  the 
Helmet-Mounted  Display  (HMD) ,  which  consists  of  a  1  inch  in 
diameter  display  attached  to  the  helmet.  The  HMD  is  a 
monocular  display  that  enables  the  crewmember  to  cross-check 
flight  and  weapons  information  superimposed  on  infrared 
sensor  imagery  while  directing  his  attention  outside  the 
cockpit.  All  the  displays  provide  the  crewmember  with  a  30° 
(vertical)  by  40°  (horizontal)  field  of  view. 


PNVS  Flight  Symbology 

The  PNVS  flight  symbology  set  consists  of  27  alpha¬ 
numeric  and  shape  coded  symbols  that  are  designed  to  help  the 
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crewmember  fly  the  aircraft.  Many  of  the  computer  generated 
symbols  are  adaptations  of  traditional  electromechanical 
instruments  and  are  located  in  fixed  positions  on  the 
displays  (e.g.,  Heading  Scale,  Vertical  Altitude  Scale). 

Some,  however,  are  unique  dynamic  representations  of  spatial 
information  that  move  about  the  displays  and  in  or  out  of  the 
viewing  areas  as  a  result  of  sensor  orientation  or  changes  in 
aircraft  position  (e.g.,  Cued  Line  of  Sight  [LOS]  Reticle, 
Hover  Position  Box) . 

To  reduce  clutter  and  to  make  the  symbolic  information 
more  task  specific,  there  are  four  operating  modes  that  pre¬ 
sent  subsets  of  the  27  symbols.  Symbols  representing  air¬ 
craft  heading,  airspeed,  altitude,  engine  torque,  and  certain 
other  basic  flight  information  are  provided  constantly  during 
all  four  modes.  The  hover  mode  adds  a  velocity  vector  and  an 
acceleration  cue  to  aid  the  pilot  in  maintaining  a  hover. 
Selection  of  the  transition  mode  adds  a  horizon  line  to  the 
hover  mode  subset  and  is  used  when  changing  from  a  hover  to 
cruise  flight.  Once  cruise  flight  has  been  established, 
selection  of  the  cruise  mode  removes  the  velocity  vector  and 
acceleration  cue  from  the  transition  mode  set.  To  aid  the 
pilot  in  returning  to  a  chosen  location  or  remaining  over  the 
location  with  a  specific  heading,  a  bob-up  mode  adds  the 
velocity  vector,  acceleration  cue,  command  heading,  and  hover 
position  symbols  to  the  basic  flight  information. 


TADS  _Wga£.dn_§ymbc>lggy 

The  TADS  weapon  symbology  set  consists  of  17  alpha¬ 
numeric  and  shape  coded  symbols.  Fourteen  symbols  are  common 
to  both  the  flight  and  weapon  symbology  sets.  The  symbols 
are  designed  to  assist  the  crewmember  during  *  ;.e  operation  of 
the  weapon  systems.  There  is  only  one  operatir.  ode  for  the 
TADS  symbology,  but  not  all  the  symbols  will  appear  on  the 
displays  at  the  same  time.  The  number  of  symbols  displayed 
at  any  given  time  depends  on  the  nature  of  the  tasks  required 
to  operate  the  weapons . 


Training  -Er.Qce.na 

To  become  fully  qualified  in  the  AH-64A  attack  heli¬ 
copter,  a  student  aviator  must  learn  to  identify  and  inter¬ 
pret  the  individual  symbols  presented  on  the  helicopter's 
visual  displays  and  to  interpret  the  information  provided  by 
groups  of  symbols.  During  the  AH-64A  Aircraft  Qualification 
Course  (AQC) ,  student  aviators  are  taught  to  use  the  symbol¬ 
ogy  through  classroom  lectures,  videotape  presentations, 


self-study  handouts,  and  technical  manuals  containing  static 
diagrams  of  the  symbology.  Opportunities  for  additional 
practice  with  the  display  symbology  are  available  on  three 
training  devices:  the  TADS  Selected  Task  Trainer  (TSTT) ,  the 
Cockpit,  Weapons,  and  Emergency  Procedures  Trainer  (CWEPT) , 
and  the  Combat  Mission  Simulator  (CMS) . 

The  TSTT  is  a  part-task  trainer  designed  to  support 
initial  CPG  qualification,  CPG  refresher  training,  and  TADS 
skill  sustainment;  it  provides  practice  only  with  weapon 
symbology.  The  CWEPT  is  a  full-scale  pilot  and  CPG  proce¬ 
dures  trainer  that  is  used  to  train  normal  and  emergency 
flight  procedures  and  avionics  equipment  operation.  The  CMS 
is  a  six  degree-of-f reedom,  motion-based  simulator  that 
provides  training  in  combat  mission  scenarios  during  the 
Combat  Skills  phase  of  the  AH-64  AQC  and  during  operational 
aviation  unit  training.  Both  the  CWEPT  and  the  CMS  provide 
the  opportunity  for  practice  with  flight  and  weapon 
symbology . 


Need 

The  training  design  features  of  the  TSTT,  CWEPT,  and  CMS 
do  not  include  training  on  basic  symbology  identification  and 
interpretation.  Students  assigned  to  training  lessons  on 
these  devices  are  assumed  to  be  familiar  with  flight  and 
weapon  symbology.  However,  TSTT,  CWEPT,  and  CMS  instructors 
report  spending  an  excessive  amount  of  time  training  basic 
symbology  skills  in  the  training  devices.  Furthermore, 
students  training  in  the  TSTT,  CWEPT,  or  CMS  typically  do  not 
have  opportunities  to  use  the  AH-64A  display  symbology  under 
the  full  range  of  missions,  modes,  weapons,  system  options, 
and  system  failures.  A  device  that  provides  specialized 
training  on  basic  symbology  identification  and  interpretation 
would  improve  the  efficiency  of  TSTT,  CWEPT,  and  CMS  instruc¬ 
tion.  Therefore,  the  Training  and  Doctrine  Command  System 
Manager  for  the  AH-64A  requested  that  the  Army  Research 
Institute  Aviation  Research  and  Development  Activity 
( ARIARDA)  develop  a  training  module  for  the  AH-64A  flight  and 
weapon  symbology. 


Project  Objectives 

ARIARDA  established  the  following  nine  design  objectives 
for  the  training  module: 

•  designed  in  a  self-instructional  format, 

•  designed  for  use  in  a  classroom  setting. 
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•  designed  to  train  symbology  for  the  full  range  of 
aircraft  mission  and  weapon  system  options, 

•  capable  of  storing  performance  data  and  providing  one 
or  more  performance  indexes  for  each  training 
exercise, 

•  capable  of  providing  immediate  feedback  and  remedial 
instruction  when  errors  occur, 

•  suitable  for  both  skill  acquisition  training  in  an 
institutional  setting  and  skill  sustainment  training 
in  an  operational  unit  setting, 

•  flexible  enough  to  allow  revisions  resulting  from  (a) 
design  changes  in  the  aircraft  or  avionics  system  and 
(b)  deficiencies  in  the  training  module  revealed  by 
formal  evaluation  and  feedback  from  the  user, 

•  designed  to  augment  rather  than  replace  existing 
training  devices,  and 

•  economical  to  develop  and  operate. 


Training  Module  Development 

Researchers  at  Anacapa  Sciences,  Inc.,  began  work  on  the 
training  module  in  December  1986.  The  researchers  inter¬ 
viewed  subject  matter  experts  (SMEs)  who  were  knowledgeable 
about  the  AH-64A  display  symbology,  including  AH-64  AQC 
academic  instructors,  CWEPT  instructors,  and  AH-64A  instruc¬ 
tor  pilots.  As  a  result  of  these  interviews,  performance 
deficiencies  were  identified  in  the  following  areas: 

•  identifying  and  interpreting  individual  symbols 
presented  alone, 

•  identifying  and  interpreting  symbols  in  the  context  of 
other  symbols, 

•  interpreting  the  meaning  of  symbology  movement, 

•  correctly  associating  switch  actions  and  control 
movements  with  static  or  dynamic  symbology,  and 

•  alternating  attention  between  the  display  symbology 
and  the  external  visual  scene. 

The  development  of  the  symbology  training  module  was 
divided  into  two  parts.  The  first  part,  the  Symbology  Tutor, 
addresses  the  first  three  performance  deficiencies  by  pro¬ 
viding  skill  acquisition  training  in  symbol  identification 
and  interpretation  for  individual  symbols  and  small,  related 
subsets  of  symbols.  The  second  part  addresses  the  last  two 
deficiencies  by  providing  training  in  (a)  correctly  asso¬ 
ciating  switch  actions  and  control  movements  associated  with 
the  symbology  and  (b)  alternating  attention  between  the 
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symbology  and  real-world  and  infrared  visual  imagery.  After 
considering  the  project  objectives,  the  training  module 
design  objectives  established  by  ARIARDA,  and  the  capa¬ 
bilities  and  limitations  of  existing  training  devices,  the 
researchers  concluded  that  the  most  appropriate  medium  for 
the  training  module  was  computer-based  instruction. 


Symbology  Tutor  Design 

The  Symbology  Tutor  is  organized  into  three  major 
sections:  (a)  an  introductory  section,  (b)  a  help  system 

that  can  be  accessed  from  anywhere  in  the  Symbology  Tutor, 
and  (c)  five  self-contained  lessons.  The  introductory 
section  provides  the  student  with  a  brief  orientation  to  the 
Symbology  Tutor  and  contains  instructions  on  how  to  use  the 
program's  features.  The  help  system  provides  the  student 
with  on-line  assistance  in  using  the  Symbology  Tutor.  The 
help  system  includes  (a)  an  overview  of  the  entire  symbology 
training  module  and  the  types  of  help  available,  (b)  a  list 
of  the  contents  of  the  five  lessons,  (c)  an  acronym  glossary, 

(d)  a  symbol  dictionary,  and  (e)  a  symbology  mode  dictionary. 
The  five  lessons  consist  of  tutorials  and  quizzes  covering 
the  following  symbols:  (a)  position/movement,  (b)  attitude/ 
altitude,  (c)  heading/navigation,  (d)  cueing/reference,  and 

(e)  weapon  delivery. 


Project  Termination 

ARIARDA  directed  Anacapa  Sciences  to  terminate  work  on 
the  AH-64A  Display  Symbology  Training  Module  project  at  the 
end  of  December  1987  because  of  funding  constraints  and  other 
project  priorities.  At  that  time,  a  draft  version  of  the 
storyboards  for  the  tutorials  and  quizzes  for  Lessons  1 
through  5  had  been  completed  and  computer  programs  for  the 
introductory  section,  the  help  system,  and  the  tutorials  and 
quizzes  for  Lessons  1  and  2  had  been  written.  The  Symbology 
Tutor  programs  were  written  in  Microsoft  QuickBASIC  and  were 
designed  to  run  on  a  Zenith  PC  AT-compat ible  microcomputer 
equipped  with  one  megabyte  of  random  access  memory,  a  hard 
disk  with  at  least  2  megabytes  available,  an  enhanced 
graphics  adapter  (EGA) ,  and  a  high  resolution  EGA  color 
monitor . 


Report  Preparation 

During  the  current  contract  year,  a  draft  project  report 
was  prepared  and  submitted  to  ARIARDA.  The  report  describes 
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the  background,  research  approach,  and  contents  of  the  Sym¬ 
bology  Tutor  and  provides  copies  of  the  storyboards  for  the 
five  lessons.  Following  a  review  by  ARIARDA,  the  report  was 
revised  and  resubmitted  in  final  form  (Ruffner,  Coker,  & 
Weeter,  1989) .  A  floppy  disc  copy  of  the  computer  programs 
that  had  been  developed  by  the  project  termination  date  was 
also  delivered. 


Work  Projected 

Submission  of  the  draft  report  and  computer  programs  to 
ARIARDA  completed  Anacapa's  work  on  this  project. 


Reference 

Ruffner,  J.  W.,  Coker,  G.  W.,  &  Weeter,  R.  D.  (1989) . 

Development  of  the  AH-64A  display  ■■■.symbology  training 
module  (Research  Note  ASI690-322-89) .  Fort  Rucker,  AL : 
Anacapa  Sciences,  Inc. 
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DEVELOPMENT  OF  THE  BASIC  MAP  INTERPRETATION  AND 
TERRAIN  ANALYSIS  COURSE  (MITAC) 

Dr.  Dudley  J.  Terrell,  Project  Director 


Background 

In  the  modern  battlefield,  Army  aviators  will  fly  at 
extremely  low  altitudes  to  avoid  detection  by  enemy  elec¬ 
tronic  sensors .  To  maintain  obstacle  clearance  while 
remaining  masked  by  terrain  features,  constant  vigilance 
outside  the  cockpit  must  be  maintained  during  low  altitude 
flight.  Momentary  shifts  of  attention  to  displays,  switches, 
and  maps  inside  the  cockpit  must  be  executed  rapidly  and 
efficiently.  Consequently,  low  altitude  navigation  requires 
superior  skill  in  map  interpretation  and  terrain  analysis. 

To  remain  geographically  oriented,  a  pilot  must  be  able  to 
glean  crucial  map  information  during  brief  glances  inside  the 
cockpit  and  to  associate  that  information  with  the  rapidly 
changing  terrain  outside  the  cockpit. 

Traditional  methods  of  low  altitude  navigation  training 
have  been  unsatisfactory  (Fineberg,  Meister,  &  Farrell,  1978/ 
Gainer  &  Sullivan,  1976;  McGrath,  1976).  Therefore,  the  U.S. 
Army  Research  Institute  Aviation  Research  and  Development 
Activity  ( ARIARDA)  has  conducted  research  to  address  the  low 
altitude  navigation  training  deficiency.  In  1976,  Anacapa 
Sciences,  under  contract  to  ARIARDA,  developed  the  Map 
Interpretation  and  Terrain  Analysis  Course  (MITAC) .  The 
MITAC  used  photographic  slides  and  motion  picture  films  of 
terrain  features  and  map  segments  to  teach  low  altitude 
navigation  skills  to  helicopter  pilots  in  a  classroom  format . 
Subsequently,  the  course  was  revised  to  an  individualized 
training  format.  Holman  (1978a,  1978b)  demonstrated  the 
effectiveness  of  the  revised  course  by  showing  that  MITAC- 
trained  student  pilots  and  enlisted  aerial  observers 
navigated  at  twice  the  speed  and  with  one-third  the  errors 
committed  by  traditionally  trained  aviators. 

In  1979,  Kelley  developed  13  additional  cinematic 
exercises  to  provide  supplemental  training  in  map  interpre¬ 
tation  and  terrain  analysis  over  a  wider  range  of  geographic 
regions  and  climates.  Each  exercise  consists  of  a  film  taken 
from  the  front  window  of  a  helicopter  flying  a  route  at  low 
altitude.  The  exercises  include  flights  over  various  geo¬ 
graphic  regions  (e.g.,  Kentucky,  Idaho,  Arizona,  and  Germany) 
with  both  snow-covered  terrain  and  summer  foliage.  The 
supplemental  exercises,  termed  the  Advanced  MITAC,  were 
subsequently  upgraded  to  a  computer-based  interactive 
videodisc  format  (Miles  &  LaPointe,  1986).  Terrell  (1989a) 
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found  that  a  significantly  greater  proportion  of  Advanced 
MITAC  students  than  control  (no  MITAC)  students  performed 
perfectly  (i.e.,  no  deviations  from  the  prescribed  route) 
during  the  post-training  navigation  test. 


Need 

Conversion  of  the  Advanced  MITAC  exercises  to  an  inter¬ 
active  videodisc  format  resulted  in  an  easy-to-use  and  effec¬ 
tive  part-task  trainer.  However,  the  material  and  equipment 
for  the  original  "Basic"  MITAC  (e.g.,  35-mm  slides,  booklets, 
projectors,  tape  players,  etc.)  are  difficult  to  use  and  are 
unsuitable  for  computer-based  training.  A  computer-based 
course  that  presents  basic  principles  of  map  interpretation 
is  needed  to  serve  as  a  prerequisite  to  the  Advanced  MITAC 
and  for  use  in  the  ARIARDA  research  program. 


Project  Objectives 

The  goal  of  this  project  is  to  develop  a  videodisc/ 
computer-based  Basic  MITAC.  This  goal  is  divided  into  the 
following  two  project  objectives: 

•  develop  the  Basic  MITAC  videodiscs  and 

•  design  experimental  courseware  for  the  Basic  MITAC. 


Development  Approach 

The  project  is  divided  into  two  phases  that  correspond 
to  the  two  project  objectives.  In  Phase  1,  the  material  from 
the  former  versions  of  Basic  MITAC  will  be  compiled  and 
upgraded  for  videodisc  development.  In  Phase  2,  instruc¬ 
tional  strategies  will  be  planned  nd  software  written  to 
implement  the  strategies.  Training  effectiveness  evaluations 
of  the  Basic  MITAC  instructional  strategies  will  be  conducted 
as  a  separate  project  (see  Training  Effectiveness  of  Aviation 
Part-Task  Trainers,  pp .  111-117  of  this  report). 


Work  Completed 


Phase  1 

The  development  of  the  Basic  MITAC  videodiscs  was  com¬ 
pleted  in  three  stages:  production,  post  production,  and 
duplication.  During  the  production  stage,  researchers  at 
Anacapa  Sciences  composed  the  narrative  for  the  Basic  MITAC, 
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selected  video  material  to  supplement  the  narrative,  and 
developed  a  script  containing  the  narrative  and  instructions 
for  taping  the  narrative  and  video  material. 

During  the  post-production  stage.  Video  Technics  of 
Atlanta,  Georgia,  recorded  the  narrative  on  audio  tape, 
generated  the  computer  graphics  and  animation,  digitized  the 
video  material,  edited  the  video  material  on  3/4-inch 
videotape,  and  dubbed  the  audio  and  video  on  1-inch  master 
videotape.  During  the  duplication  stage.  Optical  Recording 
Project/3M  in  St.  Paul,  Minnesota,  generated  a  master  video¬ 
disc  from  the  1-inch  master  tape  and  produced  videodisc 
copies  from  the  master  videodisc.  For  more  information  about 
Phase  1  of  this  project,  see  Terrell  (1989b)  or  Terrell  and 
Miles  (1989) . 


Phase  2 

During  the  current  contract  year,  work  began  on  devel¬ 
oping  experimental  software  for  the  Basic  MITAC.  Two 
programs  were  designed  and  developed  for  presenting  edited 
information  from  the  videodiscs.  One  program  is  for  dual¬ 
screen  viewing  of  the  videodisc  contents.  The  native  monitor 
of  a  host  computer  is  used  to  display  program  instructions 
and  content  menus,  and  the  second  monitor  is  used  to  display 
video  from  the  videodisc  player.  The  dual-screen  program  was 
loaned  to  the  NASA  Ames  Research  Center  to  use  in  research  on 
emergency  medical  service  flight  navigation. 

The  second  program  is  identical  to  the  first  program  in 
content,  but  it  uses  a  single  switching  monitor  and  requires 
a  special  video  overlay  card  in  the  computer.  The  single¬ 
screen  program  is  currently  being  field  tested  in  the  RS-1 
Strike  Rescue  Course  at  the  Naval  Air  Station  in 
Jacksonville,  Florida. 


Work  Projected 

Phase  1  of  this  project  is  complete.  The  development  of 
experimental  courseware  in  Phase  2  has  been  postponed  until  a 
comprehensive  plan  of  research  on  computer-based  instruc¬ 
tional  strategies  is  written  (see  Training  Effectiveness  of 
Aviation  Part-Task  Trainers,  pp .  141-149  of  this  report). 

Upon  completion  of  the  research  plan,  Basic  MITAC  programs 
will  be  developed  to  conduct  experimental  evaluations  of 
several  strategies  and  tactics  of  computer-based  instruction. 
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SURVEY  OF  RESEARCH  IN  COMPUTER-BASED 
INSTRUCTIONAL  STRATEGIES 

Dr.  Dudley  J.  Terrell,  Project  Director 


Background 

Experimental  research  about  computer-based  instructional 
strategies  is  a  recent  development  from  two  diverse  areas: 
learning  psychology  and  computer  technology  (for  a  review, 
see  Eberts  &  Brock,  1987)  .  Psychology  became  involved 
through  the  application  of  learning  research  to  the  design  of 
early  teaching  machines  (Benjamin,  1988) .  A  growing  interest 
in  the  application  of  computer  technology  to  education  has 
resulted  in  a  new  field  of  research  and  development  called 
instructional  design  (Gagne,  Briggs,  &  Wager,  1988) . 

Developments  in  computer-based  instruction  (CBI)  promise 
to  increase  training  effectiveness  by  capitalizing  on  the 
efficiency  of  the  individual  learning  process  while  reducing 
human  instructor  time.  Recognizing  this  promise,  the  Depart¬ 
ment  of  Defense  has  promoted  CBI  research  through  the  Army 
Research  Institute,  the  Army  Project  Manager  for  Training 
Devices,  the  Air  Force  Human  Resources  Laboratory,  the  Navy 
Personnel  Research  and  Development  Center,  and  the  Naval 
Training  Equipment  Center  (Dallman  et  al.,  1983;  O'Neil  & 
Evans ,  1983) . 


Need 

Much  of  the  current  CBI  research  consists  of  individual 
attempts  to  apply  recent  hardware  and  software  developments 
(e.g.,  interactive  videodisc,  high-speed  personal  computers, 
computer  graphics)  to  existing  training  programs.  Because 
these  efforts  lack  a  comprehensive  plan  for  systematic 
experimentation,  the  research  solves  immediate  training 
problems  but  fails  to  provide  information  about  the  general 
utility  of  training  technology  and  the  most  cost-effective 
methods  for  addressing  future  training  needs.  An  efficient 
program  of  CBI  research  requires  a  comprehensive  plan  that 
addresses  the  following  questions. 

•  What  is  the  best  method  of  identifying  the  critical 
learning  objectives  for  the  proposed  training  program? 

•  What  are  the  underlying  learning  principles  that  can 
be  applied  during  the  design  and  development  of  the 
proposed  training  program? 

•  What  are  the  best  media  for  presenting  the  proposed 
training  program? 
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The  U.S.  Army  Research  Institute  Aviation  Research  and 
Development  Activity  (ARIARDA)  is  developing  a  plan  to  inte¬ 
grate  existing  part-task  training  effectiveness  research 
projects  and  to  identify  CBI  research  issues.  To  support  the 
development  of  this  comprehensive  research  plan,  ARIARDA 
tasked  Anacapa  to  conduct  a  survey  of  past  research  on 
learning  principles  and  their  application  during  the  design 
of  CBI  programs. 


Project  Objectives 

The  goal  of  this  project  is  to  generate  research  ques¬ 
tions  that  can  be  addressed  in  ARIARDA  part-task  training 
effectiveness  projects.  This  goal  is  divided  into  the 
following  two  project  objectives: 

•  survey  the  research  in  computer-based  instructional 
strategies  and 

•  organize  the  literature  to  facilitate  the  development 
of  a  research  plan  for  CBI  in  aviation  training. 


Research  Approach 

To  meet  these  objectives,  Anacapa  personnel  will  conduct 
a  comprehensive  survey  of  research  on  the  application  of 
learning  principles  to  computer-based  instructional  design. 
The  project  is  divided  into  three  phases.  In  Phase  1,  the 
published  research  on  computer-based  instructional  strategies 
will  be  compiled  into  a  data  base.  In  Phase  2,  the  research 
results  will  be  evaluated  for  their  immediate  applicability 
to  computer-based  instructional  design.  In  Phase  3,  a  list 
of  questions  will  be  generated  to  guide  future  research  in 
CBI . 


Work  Completed 


Phase  1 

A  data  base  of  CBI  research  literature  was  developed 
during  the  second  contract  year  (Terrell,  1989a)  .  Although 
this  phase  is  formally  completed,  the  project  director  has 
continued  to  review  newly  published  literature  and  to  enter 
the  new  information  into  the  data  base. 
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During  the  current  contract  year,  a  critical  review  of 
the  literature  was  conducted  to  examine  the  empirical 
research  support  for  the  numerous  guidelines  and  recommenda¬ 
tions  that  have  been  published  about  CBI  design  strategies. 
The  method  and  results  of  the  literature  review  are 
summarized  in  the  next  two  sections. 

Review  method.  Two  types  of  publications  were  selected 
and  reviewed:  reports  of  empirical  research  comparing  two  or 
more  CBI  strategies  and  published  guidelines  for  the  develop¬ 
ment  of  CBI .  The  guidelines  and  experiments  were  categorized 
according  to  whether  they  pertained  to  (a)  strategies  for 
presenting  instructional  material,  (b)  strategies  for  ques¬ 
tioning  and  interactivity,  or  (c)  strategies  for  programming 
response  feedback  and  remediation  procedures. 

A  draft  report  entitled  "Strategies  of  Computer-Based 
Instructional  Design:  A  Review  of  Guidelines  and  Empirical 
Research"  was  written  and  submitted  to  ARIARDA  for  review 
(Terrell,  1989b) .  Each  section  of  the  report  begins  with  a 
list  of  guidelines  for  some  aspect  of  instructional  design 
and  a  discussion  of  the  recommendations  made  by  authors  of 
the  guidelines.  Then,  the  empirical  research  relevant  to 
each  guideline  is  reviewed. 

Review  results.  Although  there  are  some  consistencies 
in  the  literature,  the  guidelines  and  research  in  computer- 
based  instructional  strategies  are  characterized  by  contra¬ 
dictions.  In  some  cases,  authors  of  instructional  design 
guidelines  contradict  each  others'  recommendations.  In  other 
cases,  the  empirical  research  contradicts  the  experts' 
recommendations.  Finally,  empirical  research  was  not  located 
to  evaluate  many  of  the  recommendations. 

Only  5  of  the  57  guidelines  reviewed  are  supported  by 
empirical  research.  There  is  consistent  agreement  among 
authors  and  empirical  evidence  that  CBI  should  present  ques¬ 
tions,  corrective  feedback  for  incorrect  responses,  and 
multiple  trials  for  items  that  are  answered  incorrectly. 
Although  some  authors  do  not  agree  that  CBI  should  present 
prelesson  questions  or  that  adaptive  programming  should  be 
used  to  control  the  number  of  trials,  the  results  of  the 
empirical  research  indicate  that  these  strategies  are 
effective . 


Although  the  experimental  evidence  is  not  sufficient  to 
justify  complete  rejection,  8  of  the  57  guidelines  are  con¬ 
tradicted  by  the  results  of  empirical  research.  Empirical 


research  generally  contradicts  recommendations  to  use 
graphics  often,  to  provide  increasingly  informative  feedback 
for  successive  errors,  to  train  under  mild  speed  stress,  to 
present  information  before  practice,  to  randomize  the 
sequence  of  material,  to  present  part-task  training  prior  to 
whole-task  training,  to  permit  the  student  to  control  the 
presentation  of  reviews,  and  to  program  the  computer  to 
control  the  presentation  of  reviews. 

Finally,  44  of  the  guidelines  are  insufficiently 
addressed  by  empirical  research.  For  many  of  the  guidelines, 
either  the  empirical  research  produced  mixed  results  and 
further  research  is  required,  the  empirical  research  is 
inconclusive  because  of  inadequate  experimental  designs,  or 
the  empirical  research  simply  was  not  located.  Some  of  the 
guidelines,  however,  are  self-evident  and  do  not  require 
empirical  support.  Other  guidelines  are  stated  in  such 
general  terms  that  it  is  impossible  to  evaluate  their  use¬ 
fulness.  However,  some  of  the  guidelines  are  very  specific 
and  their  validity  and  generality  should  be  determined 
empirically . 


Fha.se.,  .3, 

In  Phase  3  of  this  project,  guidelines  were  selected 
that  most  urgently  require  experimental  investigation  and 
might  be  appropriate  for  the  ARIARDA  part-task  training 
effectiveness  research.  These  guidelines  and  research  issues 
are  summarized  in  the  following  paragraphs. 

Generality  of  guidelines.  The  Phase  2  survey  of  litera¬ 
ture  identified  at  least  three  important  dimensions  along 
which  generality  may  vary:  (a)  training  format  (e.g., 

computer-based  tutorials,  drills,  or  simulations),  (b)  train¬ 
ing  objectives  (e.g.,  acquisition  vs.  sustainment  training), 
and  (c)  target  population  (e.g.,  high  school  or  college 
students) .  Research  is  needed  to  examine  the  generality  of 
instructional  design  guidelines  across  these  dimensions. 

Stimuli .  Many  of  the  guidelines  for  presenting  instruc¬ 
tional  stimuli  depend  on  the  nature  of  the  lesson  content  and 
the  three  dimensions  of  generality  discussed  previously. 
However,  the  effectiveness  of  animation  and  high-quality 
graphics  should  be  empirically  demonstrated  because  of  the 
expense  that  may  be  incurred  to  employ  them. 

Interactivity .  Although  it  is  generally  accepted  that 
student-computer  interactivity  is  a  critical  element  in  CBI, 
several  questions  remain  unanswered  about  interactivity.  The 
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primary  research  issues  concern  the  type,  frequency,  and 
placement  of  training  questions  in  computer-based  tutorials. 


Feedback .  Finally,  the  available  research  suggests  that 
feedback  for  incorrect  responses  should  be  informative  and 
should  be  appropriate  to  the  type  of  error  committed.  How¬ 
ever,  relatively  little  is  known  about  remediating  repeated 
errors,  feedback  for  correct  responses,  and  response-feedback 
latency.  Empirical  research  is  required  to  develop  valid 
guidelines  for  these  issues  in  feedback  and  remediation. 


Work  Projected 

The  research  issues  identified  in  this  project  will  be 
incorporated  into  a  research  plan  for  aviation  part-task 
training  (see  Training  Effectiveness  of  Aviation  Part-Task 
Trainers,  pp .  111-117  of  this  report).  Unless  revisions  are 
required  to  the  draft  report  (Terrell,  1989b) ,  no  further 
work  is  projected. 
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TRAINING  EFFECTIVENESS  OF  AVIATION  PART-TASK  TRAINERS 
Dr.  Dudley  J.  Terrell,  Project  Director 


Background 

Many  aviation  tasks,  procedures,  and  skills  are  train- 
able  with  the  use  of  simulators  and  part-task  training 
devices  (Flexman  &  Stark,  1987)  .  The  development  of  this 
training  technology  requires  knowledge  of  the  psychological 
principles  underlying  the  individual  learning  processes  and 
of  effective  instructional  strategies  for  various  kinds  of 
performance.  To  acquire  this  knowledge,  researchers  at  the 
U.S.  Army  Research  Institute  Aviation  Research  and  Develop¬ 
ment  Activity  (ARIARDA)  have  identified  several  operational 
training  problems  at  the  U.S.  Army  Aviation  Center  and  have 
developed  prototype  training  devices  to  address  these 
problems  (Miles  &  LaPointe,  1986a,  1986b;  Ruffner,  Coker,  & 
Weeter,  1989;  Terrell  &  Miles,  1989) .  The  prototype  training 
devices  are  designed  to  serve  two  purposes:  (a)  to  remediate 
existing  training  deficiencies  and  (b)  to  be  used  in 
computer-based  instructional  strategies  research  at  ARIARDA. 


Need 

The  development  of  prototype  devices  for  training 
research  is  an  evolutionary  process.  Preliminary  task 
analyses,  subject  matter  expert  advice,  and  existing  research 
are  used  to  design  the  first  systems.  At  several  stages  in 
the  design  process,  the  devices  must  be  subjected  to  training 
effectiveness  analyses.  The  results  of  these  analyses  are 
used  to  improve  the  design  of  the  training  system.  Experi¬ 
ments  designed  to  evaluate  the  effectiveness  of  various 
instructional  strategies  provide  the  basis  for  making 
recommendations  about  the  optimal  use  of  the  training 
systems . 


Research  Objectives 

The  general  purpose  of  this  research  area  is  to  evaluate 
the  effectiveness  of  prototype  training  devices  and  instruc¬ 
tional  strategies  being  developed  at  ARIARDA.  Specifically, 
the  following  devices  will  be  evaluated  to  determine  their 
training  effectiveness  and  to  elucidate  generally  effective 
computer-based  instructional  strategies: 

•  the  Advanced  Map  Interpretation  and  Terrain  Analysis 
Course  (MITAC) , 
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•  the  Modernized  Cobra  Preflight  Inspection  Trainer,  and 

•  the  Basic  MITAC. 


Research  Approach 

A  comprehensive  plan  of  research  will  be  developed  to 
maximize  the  generality  of  results  from  experiments  with  the 
Advanced  MITAC,  the  Modernized  Cobra  Preflight  Inspection 
Trainer,  the  Basic  MITAC,  and  other  prototype  training 
devices  that  are  developed  by  ARIARDA.  The  evaluation  of 
each  training  device  will  be  conducted  as  a  separate  project 
under  this  research  area.  Because  the  current  projects  are 
at  different  stages  of  development,  a  general  research 
approach  is  presented.  The  status  of  the  research  is 
described  separately  for  each  project. 

To  evaluate  the  effectiveness  of  computer-based  instruc¬ 
tional  strategies  with  the  ARIARDA  part-task  training 
devices,  the  following  tasks  must  be  accomplished: 

•  design  and  conduct  training  effectiveness  experiments 
for  each  prototype  training  device, 

•  use  the  experimental  results  to  revise  the  courseware 
design,  and 

•  continue  training  effectiveness  experimentation  until 
the  maximum  training  benefit  is  realized. 


Development  of  Part-Task  Training  Effectiveness 

Research  Plans 


Work  Completed 

During  the  current  contract  year,  researchers  at  ARIARDA 
and  Anacapa  Sciences  began  the  development  of  an  integrative 
research  plan.  Three  categories  of  training  effectiveness 
research  questions  were  identified.  In  the  first  category, 
the  application  of  instructional  principles,  the  broad 
research  question  is  how  can  aviation  skills  be  trained  more 
effectively  with  a  microcomputer  than  with  the  current 
training  method.  To  answer  this  question,  research  will  be 
conducted  (a)  to  determine  which  aviation  skills  are  required 
to  perform  missions,  (b)  to  estimate  the  effectiveness  of 
current  institutional  and  unit  training  of  these  skills,  and 
(c)  to  explore  methods  of  computer-based  instruction  to 
improve  the  effectiveness  and  efficiency  of  this  training. 

The  second  category  of  questions  is  designed  to  deter¬ 
mine  the  best  performance  taxonomy  for  conducting  aviation 
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part-task  training  research.  Current  military  training 
research  uses  a  variety  of  taxonomic  approaches,  including 
cognitive  (e.g.,  skills,  abilities,  knowledge),  behavioral 
(e.g.,  responses,  response  classes,  response  chains),  and 
workload  terms  (e.g.,  tasks,  functions,  segments,  missions). 
Research  will  be  conducted  to  adapt  or  develop  a  taxonomy  of 
aviation  performance  for  use  in  the  ARIARDA  part-task 
training  effectiveness  evaluations. 

For  example,  in  July  1989,  researchers  at  Anacapa 
Sciences  proposed  the  development  of  a  computer-based 
measurement  procedure  for  map  interpretation  and  terrain 
analysis  skills  and  the  empirical  evaluation  of  the  reli¬ 
ability,  validity,  and  sensitivity  of  the  procedure.  The 
measurement  procedure  will  serve  as  a  criterion  measure  for 
training  effectiveness  research  with  the  Advanced  (and  Basic) 
MITAC  and  as  a  diagnostic  tool  for  operational  training  in 
map  interpretation  and  terrain  analysis. 

The  third  category  of  questions  addresses  the  condi¬ 
tional  circumstances  that  affect  aviation  performance;  that 
is,  how  do  variations  in  external  conditions  affect  the 
performance  of  aviation  tasks.  Preliminary  research  will 
outline  the  possible  external  conditions  of  aviation  tasks 
and  the  dimensions  along  which  these  conditions  typically 
vary.  Formal  research  will  investigate  the  effects  of  such 
variations  on  aviation  performance  and  will  develop  strate¬ 
gies  for  incorporating  effective  variations  into  part-task 
training . 


Work  Projected 

The  broad  research  questions  will  be  developed  and 
refined  into  specific  experimental  and  analytical  objectives. 
For  each  objective,  a  plan  of  research  will  be  drafted  and 
submitted  to  ARIARDA. 


Advanced  MITAC  Interactive  Videodisc  Training 

The  original  format  for  the  Advanced  MITAC  training 
program  was  a  set  of  16-mm  film  exercises  in  low-altitude 
geographic  orientation  (Kelley,  1979) .  The  16-mm  films  were 
converted  to  videodisc  and  the  exercises  were  upgraded  to  an 
interactive  computer-based  training  format  (Miles  &  LaPointe, 
1986b) .  During  the  current  contract  year,  the  Advanced  MITAC 
software  was  converted  from  QuickBASIC  to  Turbo  Pascal  and 
modified  to  run  on  the  Electronic  Information  Delivery  System 
(EIDS) .  The  Advanced  MITAC  for  EIDS  includes  the  standard 
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courseware,  a  data  collection  program,  and  an  instructor’s 
review  program.  The  interface  permits  the  use  of  the  EIDS 
keypad,  keyboard,  or  lightpen. 


In  addition,  a  dual-screen  version  of  the  Advanced  MITAC 
was  developed.  The  native  monitor  of  a  host  computer  is  used 
to  display  program  instructions  and  to  receive  student  input. 
A  second  monitor  is  used  to  display  video  from  the  videodisc 
player . 

The  current  research  and  development  effort  with  the 
Advanced  MITAC  focuses  on  (a)  the  effectiveness  of  the  inter¬ 
active  videodisc  method  for  training  geographic  orientation 
skills  and  (b)  the  evaluation  of  computer-based  instructional 
strategies . 


Work  Completed 

An  experiment  was  conducted  to  evaluate  the  training 
effectiveness  of  the  Advanced  MITAC  and  to  compare  the 
effects  of  two  methods  of  computer-based  error  remediation  on 
inflight  navigation  performance.  The  results  suggest  that 
the  Advanced  MITAC  is  effective  for  teaching  contour-level 
navigation  skills  to  helicopter  pilots.  The  results  suggest 
that  computer-generated  error  remediation  is  more  effective 
than  student-generated  remediation,  but  further  research  is 
required  to  confirm  this  finding.  A  project  report  (Terrell, 
1989a)  and  a  plan  for  follow-on  research  (Terrell,  1988)  were 
submitted  to  ARIARDA  for  review. 

In  February  1989,  the  dual-screen  version  of  the 
Advanced  MITAC  was  delivered  to  the  NASA  Ames  Research  Center 
at  Moffett  Field,  California,  for  field  testing  in  an  emer¬ 
gency  medical  service  flight  navigation  program.  In  July 
1989,  a  set  of  Advanced  MITAC  vidediscs,  software,  and 
computer  equipment  was  delivered  to  the  Naval  Air  Station  in 
Jacksonville,  Florida,  for  field  testing  in  the  HS-1  Strike 
Rescue  Course  at  Jacksonville. 


Work  Projected 

The  use  of  Advanced  MITAC  by  the  Naval  Air  Station  and 
by  the  NASA  Ames  Research  Center  will  continue  to  be  moni¬ 
tored.  If  the  ARIARDA  part-task  training  effectiveness 
research  plan  requires  the  use  of  the  Advanced  MITAC, 
research  will  begin  to  develop  a  computer-based  performance 
measurement  procedure. 
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Modernized  Cobra  Preflight  Inspection  Trainer 


The  original  format  for  this  training  program  was  an 
interactive  videotape  controlled  by  a  microcomputer  (Miles  & 
LaPointe,  1986a) .  Preliminary  research  demonstrated  the 
effectiveness  of  the  experimental  program  by  measuring 
performance  on  a  preflight  inspection  multiple-choice  test 
(Intano,  1988) .  However,  incidental  observations  during  this 
research  suggested  that  the  videotape  format  was  less  suit¬ 
able  than  videodisc  for  computer-based  training.  The  current 
research  effort  focuses  on  the  development  of  a  videodisc 
version  of  the  training  program  and  the  evaluation  of 
computer-based  instructional  strategies  with  the  videodisc 
version  of  the  trainer. 


Work  Completed 

The  Preflight  Inspection  Trainer  videotape  was  converted 
to  a  videodisc  format,  and  the  courseware  design  and  pro¬ 
gramming  was  initiated.  Software  was  developed  for  dual¬ 
screen  and  single-screen  viewing  of  the  videodisc  contents 
(see  Terrell,  1989b) .  The  single-screen  version  of  the 
training  program  is  currently  being  field  tested  in  the 
flightline  classrooms  of  the  AH-1  Aviator  Qualification 
Course  at  Fort  Rucker,  Alabama. 


Work  Projected 

The  field  test  of  the  Preflight  Inspection  Trainer  will 
continue  to  be  monitored.  If  required  by  the  training  effec¬ 
tiveness  research  plan,  the  Preflight  Inspection  Trainer  will 
be  used  in  research  on  computer-based  instructional 
strategies . 


Basic  MITAC  Interactive  Videodisc  Training 

The  Basic  MITAC  is  currently  being  developed  by  Anacapa 
Sciences  as  a  separate  project  (see  Development  of  the  Basic 
Map  Interpretation  and  Terrain  Analysis  Course,  pp.  99-103  of 
this  report).  Upon  completion  of  that  project,  training 
effectiveness  evaluations  will  be  conducted  under  this 
research  area.  Specifically,  the  research  will  evaluate  the 
effectiveness  of  the  Basic  MITAC  in  training  general  map 
interpretation  and  terrain  analysis  skills  as  a  prerequisite 
for  Advanced  MITAC  training. 
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If  required  by  the  training  effectiveness  research  plan, 
experiments  will  be  designed  and  conducted  with  the  Basic 
MITAC  to  evaluate  the  effectiveness  of  various  instructional 
strategies.  The  strategies  will  include  different  computer- 
based  branching  routines  for  remediating  deficiencies, 
different  methods  for  presenting  drills  and  tutorials,  full- 
motion  versus  still  graphics,  and  visual  versus  audio 
narratives . 
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ACCIDENT  SCENARIO  TRAINING 
Mr.  Joseph  L.  Zeller,  Jr.,  Project  Director 

Background 

Much  of  the  training  that  Army  aviators  currently 
receive  in  both  aircraft  and  flight  simulators  is  designed  to 
enhance  safety,  but  little  of  the  training  is  designed  to 
reduce  specific  types  of  accidents.  In  most  cases,  deliber¬ 
ately  creating  accident-producing  situations  in  actual  flight 
is  too  risky  even  for  aviators  with  substantial  experience. 
There  are  several  reasons  that  little  flight  simulator  time 
has  been  devoted  to  accident  reduction  training.  Early 
flight  simulators  were  developed  as  instrument  training 
devices  and  procedural  trainers;  they  did  not  have  the  visual 
systems  needed  to  create  accident-producing  situations.  As 
more  advanced  simulators  with  both  day  and  night  visual  capa¬ 
bilities  have  been  developed,  they  have  been  used  primarily 
to  enhance  operational  and  tactical  skills.  Finally,  the 
personnel  responsible  for  developing  simulator  training 
programs  have  not  had  adequate  information  about  the  situa¬ 
tions  in  which  aircraft  accidents  occur  and  the  reasons  for 
their  occurrence. 

In  support  of  the  Army  Research  Institute  Aviation 
Research  and  Development  Activity  (ARIARDA)  accident  preven¬ 
tion  research  program,  Anacapa  Sciences  was  tasked  to  deter¬ 
mine  the  viability  of  accident  scenario  training.  Accident 
scenario  training  involves  the  use  of  a  flight  simulator  or 
other  training  device  to  reenact,  as  accurately  as  possible, 
all  the  conditions  and  actions  that  have  been  shown  to  con¬ 
tribute  to  a  frequently  occurring  type  of  accident.  In 
principle,  accident  scenario  training  may  serve  to  reduce 
accident  likelihood  in  one  or  more  of  the  following  ways: 

•  teach  aviators  to  recognize  and  avoid  specific  situa¬ 
tions  and  conditions  in  which  accident  likelihood  is 
high  ; 

•  teach  aviators  to  recover  the  aircraft  safely  when 
accident-producing  situations  and  conditions  cannot  be 
avoided;  and 

•  teach  aviators  the  specific  search  skills,  perceptual 
judgment  skills,  and  information  processing  skills 
that  are  required  to  avoid  high  hazard  situations  or 
to  function  safely  in  such  situations. 
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Research  Objectives 


The  general  objectives  of  this  project  are  to  develop 
accident  scenario  training  for  a  sample  of  accident  types  and 
to  assess  the  cost-effectiveness  of  accident  scenario 
training  for  the  target  accidents,  considered  individually 
and  collectively.  The  specific  technical  objectives  of  the 
research  project  are  as  follows: 

•  identify  types  of  helicopter  accidents  that  are  candi¬ 
dates  for  accident  scenario  training, 

•  validate  the  accident  scenario  training  concept  using 
a  small  sample  of  accident  types  (target  accidents), 

•  develop  training  scenarios  for  the  target  accidents, 
and 

•  assess  the  training  and  cost  effectiveness  of  accident 
scenario  training  aimed  specifically  at  the  target 
accidents . 


Research  Approach 

The  approach  selected  for  meeting  the  research  objec¬ 
tives  involves  three  sequential  tasks.  The  first  task  in 
assessing  the  feasibility  and  effectiveness  of  accident 
scenario  training  is  to  identify  the  types  of  helicopter 
accidents  that  occur.  This  task  will  be  accomplished  by 
conducting  a  systematic  review  of  a  large  sample  of  accidents 
for  which  human  error  is  known  to  be  a  causal  factor.  Acci¬ 
dents  will  be  classified  according  to  the  circumstances 
surrounding  the  accident  and  the  commonality  of  training 
necessary  to  eliminate  the  errors  that  caused  or  contributed 
to  the  accident . 

The  second  task  is  to  select  a  small  sample  of  accident 
types  (target  accident  types)  to  be  investigated  in  vali¬ 
dating  the  accident  scenario  training  concept.  The  selection 
criteria  will  include  the  capability  of  conducting  the 
training  in  available  simulators,  the  importance  (frequency 
and  severity)  of  the  accident  type,  and  the  representative¬ 
ness  of  the  set  of  accident  types. 

The  third  task  is  to  develop  training  scenarios  for  the 
target  accidents  and  to  conduct  research  to  assess  their 
training  and  cost  effectiveness  in  reducing  the  target  acci¬ 
dents.  The  development  of  the  training  scenarios  and  the 
specification  of  training  equipment  requirements  will  be 
performed  interactively  to  avoid  developing  scenarios  that 
impose  excessive  equipment  modifications. 
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Work  Completed 


Work  on  this  project  began  in  June  1989.  Data  for 
selected  Class  A-C  helicopter  accidents  that  occurred  since 
fiscal  year  1984  were  requested  from  the  U.S.  Army  Safety 
Center  (USASC) .  The  requested  data  are  computer  printouts  of 
selected  accident  reporting  forms  that  provide  information 
such  as  the  findings  and  recommendations,  a  narrative  account 
of  the  investigation,  data  about  the  personnel  involved 
(including  their  injuries  if  any) ,  and  the  weather  at  the 
time  of  the  accident.  At  the  end  of  the  current  contract 
year,  367  computerized  accident  cases  had  been  received  from 
the  USASC,  including  73  cases  in  which  night  vision  devices 
(NVDs )  were  in  use.  In  addition,  the  USASC  had  approved  a 
request  for  access  to  the  computerized  accident  files  (Army 
Safety  Management  Information  System — ASMIS) . 

Written  procedures  for  analyzing  and  classifying  the 
accident  cases  and  recording  the  results  were  developed  and 
refined.  The  procedures  describe  the  steps  to  be  followed  in 
reviewing  each  accident  case  file  and  the  method  for  identi¬ 
fying  the  precipitating  function  failure  (error  or  action 
that  led  directly  to  the  accident) ,  predisposing  factors 
(errors  or  actions  that  increased  the  likelihood  of  the 
accident),  and  the  phase  of  flight  in  which  they  occurred. 

Approximately  150  accident  case  files,  including  all  of 
the  cases  in  which  NVDs  were  in  use,  were  reviewed  by  at 
least  two  different  analysts.  Accidents  involving  the  Pilot 
Night  Vision  System  (PNVS)  were  deleted  from  the  sample 
because  of  the  differences  between  the  PNVS  and  the  other 
night  vision  devices  that  use  image  intensifying  tubes.  In 
addition,  accidents  that  occurred  prior  to  flight  and  those 
that  contained  insufficient  information  for  analysis  were 
deleted  from  the  review.  The  remaining  reviews  of  the 
accident  cases  were  compared  and  any  differences  in  cate¬ 
gorization  by  the  two  analysts  were  resolved.  At  the  end  of 
the  contract  year,  each  analyst  had  begun  to  develop  an 
initial  type  classification  of  accidents  that  appear  to  be 
candidates  for  accident  scenario  training. 


Work  Projected 

The  remaining  accident  case  files  will  be  reviewed  and 
the  results  compared  and  resolved.  The  initial  type 
classification  will  be  developed  and  refined  as  additional 
cases  are  analyzed.  Target  accident  types  will  be  selected 
on  the  basis  of  their  suitability  for  scenario  training  and 
specific  training  requirements  will  be  formulated  for  each 


121 


target  type.  Two  or  three  target  accident  types  will  be 
selected  and  used  to  validate  the  concept  of  accident 
scenario  training.  If  the  research  findings  indicate  that 
the  concept  is  valid,  additional  training  scenarios  will  be 
developed  and  implemented. 
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TECHNICAL  ADVISORY  SERVICE:  SUPPORT  TO  THE 
SPECIAL  OPERATIONS  AIRCRAFT  PROGRAM  MANAGER'S  OFFICE 

Mr.  Carl  R.  Bierbaum,  Technical  Advisor 


Background 

The  U.S.  Army  provides  aviation  support  to  the  Special 
Operations  Forces,  a  Department  of  Defense  unit.  The  CH-47D 
and  UH-60A  aircraft  are  currently  being  used  for  aviation 
support,  but  the  Army  is  developing  special  aircraft  that 
will  have  additional  capabilities.  The  Special  Operations 
Aircraft  (SOA)  Program  Manager's  (PM)  Office  at  the  Army 
Aviation  Systems  Command  has  been  tasked  to  develop  the 
MH-60K  and  MH-47E  aircraft.  These  aircraft  will  use  the 
existing  CH-47D  and  UH-60A  airframes  but  will  have  a  new, 
integrated  cockpit.  A  standardized,  integrated  cockpit 
featuring  four  multifunction  display  (MFD)  units  will  replace 
the  present  CH-47D  and  UH-60A  instrument  and  gauge 
configurations . 

The  MH-60K  and  MH-47E  aircraft  are  being  designed  to 
provide  special  operations  aircraft  that  have  increased  capa¬ 
bilities  and  reduced  crewmember  workload.  However,  the  high 
technology  modif ications  being  proposed  for  the  MH-60K  and 
MH-47E  cockpits  may  increase  workload  by  placing  additional 
demands  on  the  mental  resources  of  the  crewmembers. 
Researchers  from  the  U.S.  Army  Research  Institute  Aviation 
Research  and  Development  Activity  (ARIARDA)  and  Anacapa 
Sciences  have  developed  a  methodology  for  conducting 
mission/task  analyses  and  predicting  workload  for  emerging 
systems  (Aldrich,  Szabo,  &  Craddock,  1986) .  Additionally, 
Anacapa  researchers,  under  contract  to  ARIARDA,  developed  a 
UH-60  workload  prediction  model  (Bierbaum,  Szabo,  &  Aldrich, 
1989) .  Because  of  the  experience  gained  from  the  conduct  of 
these  projects,  Anacapa  was  tasked  with  providing  technical 
advisory  services  to  the  MH-60K  and  MH-47E  development 
programs . 


Support  Provided 

The  primary  technical  advisory  support  to  the  MH-60K  and 
MH-47E  development  programs  during  the  current  contract  year 
was  to  participate  in  the  fifth  Crew  Station  Working  Group 
(CSWG)  meeting  for  the  Special  Operations  Aircraft  at  the 
Boeing  Helicopter  facilities  in  Philadelphia,  Pennsylvania. 
The  major  areas  of  interest  are  summarized  below. 
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At  the  meeting,  the  Anacapa  technical  advisor  presented 
a  briefing  on  the  Task  Analysis  Workload  (TAWL)  methodology 
and  the  task  analysis  workload  prediction  models  for  the  CH- 
47,  UH-60,  MH-47E  and  MH-60K.  The  Anacapa  technical  advisor 
also  provided  input  for  the  following  cockpit  design  items: 

•  procedure  for  uploading  and  storing  navigational  aid 
frequencies , 

•  auxiliary  fuel  tank  fuel  level  sensors, 

•  types  of  pilot  and  copilot  doors  for  the  MH-60K, 

•  switches  and  procedures  to  zero  the  classified 
avionics  and  mission  processors,  and 

•  switches  to  increase  the  interphone  control  system 
(ICS)  panel  from  5  to  7  positions. 


Support  Projected 

No  further  support  is  projected  at  this  time.  However, 
Anacapa  personnel  will  resume  technical  advisory  support  to 
the  SOA  PM  if  tasked  by  ARIARDA. 
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TECHNICAL  ADVISORY  SERVICE:  SUPPORT  TO 
OPERATOR  WORKLOAD  (OWL)  RESEARCH 

Dr.  David  B.  Hamilton  and  Mr.  Carl  R.  Bierbaum, 
Technical  Advisors 


Background 

In  1986,  the  Army  Research  Institute  (ARI)  awarded  a 
multi-year  contract  (MDA903-86-C-0384 )  for  research  in 
"Controlling  Operator  Workload  in  Army  Systems  Design  and 
Evaluation"  to  Analytics,  Inc.,  of  Willow  Grove,  Pennsyl¬ 
vania.  The  contract  was  awarded  because  of  the  recognized 
need  for  systematic  and  validated  methods  to  estimate 
workload  or  to  isolate  its  causes  during  system  design. 
Specifically,  the  research  was  designed  to  accomplish  the 
following  objectives: 

•  assess  the  existing  operator  workload  measures  for 
their  utility  in  system  design  and  evaluation, 

•  evaluate  operator  workload  in  three  representative 
Army  systems, 

•  develop  guidelines  for  choosing  the  most  appropriate 
workload  assessment  techniques  for  varying  conditions, 
and 

•  prepare  a  pamphlet  and  handbooks  to  guide  personnel  in 
assessing  workload  in  emerging  Army  systems. 

To  meet  the  objectives,  operator  workload  was  examined 
in  the  AQUILA  remotely  piloted  vehicle  and  the  Forward  Area 
Air  Defense  Line  of  Sight-Forward  (Heavy)  system.  The 
research  objectives  required  the  third  system  to  be  an  avia¬ 
tion  system.  After  a  false  start  with  the  Airborne  Target 
Handover  System,  the  UH-60  Blackhawk  was  chosen  as  the 
representative  aviation  system. 

A  comprehensive  review  and  evaluation  of  workload 
methodologies  was  conducted.  The  research  concentrated  on 
several  analytic  methods  of  workload  prediction  and  the 
comparison  of  subjective  workload  measures  in  the  three 
systems.  After  reviewing  several  analytic  methods,  the 
ARIARDA/Anacapa  Sciences  task  analysis/workload  (TAWL) 
methodology  and  the  TAWL  Operator  Simulation  System  (TOSS) 
were  selected  for  evaluation  because  they  had  previously  been 
used  to  develop  the  UH-60  workload  prediction  model 
(Bierbaum,  Szabo,  &  Aldrich,  1989) .  At  the  request  of 
Analytics,  ARIARDA  tasked  Anacapa  to  support  the  research  to 
assess  the  operator  workload  in  the  UH-60  aircraft. 
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Research  Approach 


The  Analytics  research  plan  was  divided  into  three 
phases.  In  phase  one,  subjective  measures  of  workload  were 
collected  during  simulated  flight  missions  in  the  UH-60 
flight  simulator  at  Fort  Rucker,  Alabama.  In  phase  two,  a 
TAWL  workload  prediction  model  was  developed  to  predict  the 
workload  of  the  operators  during  the  simulated  mission 
scenario.  In  phase  three,  the  subjective  measures  collected 
in  phase  one  were  compared  with  the  predictions  of  the  model 
obtained  in  phase  two. 


Support  Provided 

Anacapa  personnel  provided  the  following  support  to  the 
operator  workload  research: 

•  attended  meetings  and  assisted  in  planning  the  UH-60 
simulator  research  (phase  one)  , 

•  produced  a  workload  prediction  model  for  the  UH-60 
flight  mission  (thus  completing  phase  two  in  its 
entirety) ,  and 

•  assisted  in  the  interpretation  of  the  results  found 
during  the  comparison  of  the  simulator  data  and  the 
model's  predictions. 


The  majority  of  the  Anacapa  support  was  used  to  develop 
the  Analyt ics/UH- 60  workload  prediction  model.  During  the 
UH-60  flight  simulator  phase,  Analytics  researchers  collected 
subjective  estimates  of  operator  workload  in  12  mission 
segments.  Because  each  of  the  Analytics  segments  contained 
several  Anacapa  UH-60  defined  segments,  Anacapa  personnel 
programmed  and  tested  new  decision  rules  to  model  the 
simulated  segments.  The  model  data  and  the  TOSS  software 
system  was  delivered  to  Analytics  in  April  1989. 


Subsequently,  Analytics  personnel  executed  the  model  and 
summed  the  model's  predictions  over  executions,  workload 
components,  and  time  to  produce  single  workload  estimates  for 
6  of  the  12  mission  segments.  They  scaled  the  estimates  to 
the  range  of  the  subjective  measures  (1-100)  and  correlated 
the  average  operator  workload  for  several  crews  in  the 
simulator  with  the  output  of  the  workload  prediction  model. 
Generally,  the  correlations  between  the  subjective  measures 
and  predicted  workload  were  high  (x  =  .62  to  .95) .  They 
concluded  that  the  TAWL  methodology  has  substantial  potential 
as  a  workload  estimation  technique  that  could  be  applied 
before  system  development.  The  results  of  the  research  are 
reported  in  Iavecchia,  Linton,  Bittner,  and  Byers  (1989). 
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Support  Projected 


No  further  support  is  projected  for  the  Analytics 
Operator  Workload  research. 
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TECHNICAL  ADVISORY  SERVICE:  SOFTWARE  DEVELOPMENT  FOR 
FLIGHTLINE  RESEARCH  SYSTEMS 

Ms.  Stephanie  Noland  and  Ms.  Laura  Fulford, 
Technical  Advisors 


Background 

Scheduling  of  aviation  training  resources  at  the  U.S. 
Army  Aviation  Center  (USAAVNC),  Fort  Rucker,  Alabama,  is  a 
time-consuming  and  potentially  costly  activity.  For  example, 
the  l-14th  Battalion  coordinates  and  schedules  requests  for 
aircraft,  stagefields,  refueling,  bus  and  air  transportation, 
ammunition,  and  remote  landing  site  lighting.  The  scheduling 
of  these  resources  is  accomplished  manually;  a  mistaken  order 
of  only  one  extra  aircraft  can  cost  hundreds  of  dollars. 

The  resources  required  for  all  classes  are  scheduled  in 
5-week  written  projections,  1-week  updated  written  projec¬ 
tions,  and  two  daily  squaring  sessions  conducted  by  tele¬ 
phone.  The  Army  currently  uses  Form  325  for  this  scheduling. 


Research  Objective 

The  objective  of  this  project  is  to  develop  a  computer 
hardware  and  software  system  that  will  automate  the 
scheduling  of  aviation  training  resources.  An  efficient, 
automated  system  will  increase  speed  and  accuracy  and  reduce 
the  costs  of  scheduling  errors.  In  January  1988,  the  U.S. 
Army  Research  Institute  Aviation  Research  and  Development 
Activity  ( ARIARDA)  assigned  the  development  of  the  automated 
system  to  another  of  their  contractors.  In  July  1988, 
ARIARDA  tasked  Anacapa  Sciences  to  provide  software 
development  support  to  the  project . 


Research  Approach 

The  l-14th  Battalion  at  the  USAAVNC  was  chosen  as  the 
prototype  training  organization  for  this  project  because  it 
has  che  most  complex  resource  requirements.  The  l-14th 
Battalion  has  six  training  companies  (A  -  F) ;  each  company 
manages  between  two  and  five  different  courses.  As  many  as 
seven  classes  may  be  in  residence  in  a  company  at  any  given 
time . 


A  MicroVax  II  (VAX)  microcomputer  was  chosen  as  the 
central  processor  for  the  scheduling  system.  The  six 
training  companies  and  the  battalion  staff  were  provided  with 
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Zenith  248  personal  computers  to  use  in  logging  onto  the  VAX 
and  for  accessing  the  resident  programs  and  data.  The  system 
design  provides  customized  programs  that  enable  each  company 
to  build,  edit,  update,  and  print  data  about  its  classes.  At 
the  battalion  level,  computer  programs  gather  and  organize 
the  information  from  the  six  on-line  company  class  files  into 
battalion  reports.  The  battalion  reports  include  aircraft, 
ammunition,  stagefield,  refueling,  bus  transportation,  and 
air  transportation  requests  for  each  company.  A  battalion 
program  gathers  the  twice  daily  company  change  requests  into 
a  transaction  log  to  square  with  the  brigade  schedules. 


Support  Provided 

During  the  current  contract  year,  Anacapa  continued  to 
provide  technical  assistance  in  developing  the  Flightline 
Research  System  (FLRS )  software.  In  October  1988,  the 
Anacapa  technical  advisor  was  assigned  the  role  of  systems 
manager  for  the  MicroVax  II  hardware  system  and  was 
responsible  for  completing  the  development  of  the  FLRS 
software.  She  was  also  responsible  for  training  l-14th 
personnel  and  testing  A  Company  and  Battalion  software. 

Two  new  battalion  programs  were  developed  and  tested: 
one  program  compiles  Artillery  Support  Requests  from  all 
companies  and  the  second  program  compiles  Cover  Flight 
Requests.  The  new  programs  were  linked  to  the  battalion  menu 
system  and  all  company  course  templates  were  modified  to 
include  requests  for  these  resources. 

In  November  1988,  the  Anacapa  technical  advisor 
developed  the  "ARIARDA  FLRS  Company  Level  Users  Guide"  and 
the  "ARIARDA  FLRS  Battalion  S3  Users  Guide."  The  guides 
explain  in  detail  how  to  use  each  of  the  company  and  bat¬ 
talion  programs.  Both  guides  were  reviewed  by  the  ARIARDA 
project  director  and  by  the  Battalion  Operations  Officer. 
Copies  of  the  guides  were  distributed  to  Battalion,  A 
Company,  and  F  Company  operations  personnel. 

During  December  1988,  the  company  and  battalion  programs 
were  modified  so  that  the  battalion  programs  accessed  data 
from  the  second  week  of  the  company  class  files.  This 
allowed  the  battalion  to  obtain  weekly  projections  on  the 
preceding  Tuesday  rather  than  on  Friday.  All  company  test 
files  were  modified  accordingly,  and  tests  were  run 
successfully . 

The  Anacapa  technical  advisor  trained  F  Company 
operations  personnel  to  use  all  of  the  company  programs. 

They  used  the  programs  to  build  their  classes  and  to  submit 
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their  5-week  resource  predictions.  They  did  not  use  the 
company  programs  for  daily  squaring. 


Also  during  December  1988,  the  Battalion  Operations 
Officer  requested  modifications  to  the  scheduling  program  and 
templates  so  that  18  rather  than  12  weeks  could  be  scheduled. 
This  revision  was  necessary  to  accommodate  the  80-training- 
day  IERW  course.  In  January  1989,  the  modified  software  was 
tested  and  benchmarked. 

As  a  result  of  the  tests,  the  technical  advisor 
determined  that  the  software  system  would  be  unable  to  meet 
the  deadlines  for  squaring  the  daily  schedules .  The  techni¬ 
cal  advisor  attempted  to  convert  the  VAX-resident  class 
building  programs  to  run  on  the  Zenith  248  computer  using  the 
spreadsheet  20/20  software.  However,  only  3  weeks  of  data 
could  be  installed  because  of  the  memory  limitation  of  640K. 
Because  of  these  problems,  ARIARDA  decided  to  cancel  the  A 
Company  and  Battalion  test  and  to  cease  work  on  the  Form  325 
software . 


Support  Projected 

No  additional  support  is  projected  for  the  Flightline 
Research  System. 
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TECHNICAL  ADVISORY  SERVICE:  PROGRAMMING  SUPPORT  TO 
THE  THREAT  PART-TASK  TRAINER 

Mr.  Gary  W.  Coker,  Technical  Advisor 


Background 

Force  modernization  has  resulted  in  a  proliferation  of 
complex  systems  in  the  Army.  To  train  personnel  on  these 
systems,  the  Army  has  developed  a  cost-effective,  standard¬ 
ized  method  for  delivering  doctrinal,  instructional,  and 
technical  materials.  This  method  is  called  the  Electronic 
Information  Delivery  System  (EIDS) ,  an  integrated  computer- 
videodisc  interactive  system.  A  program  called  ASSIST  is  the 
standard  courseware  authoring  system  for  use  on  EIDS. 

As  part  of  an  ongoing  research  program  in  part-task 
training  effectiveness,  the  Army  Research  Institute  Aviation 
Research  and  Development  Activity  (ARIARDA)  has  developed  an 
interactive  videodisc  (IVD)  system  to  provide  Soviet  threat 
identification  training  to  soldiers.  The  THREAT  system 
comprises  a  PC-AT  compatible  computer,  a  standard  videodisc 
player,  and  a  separate  video-graphics  overlay  controller. 

The  courseware  was  authored  using  the  Computer-Based 
Memorization  System  (CBMS) . 


Need 

The  THREAT  identification  trainer  developed  by  ARIARDA 
uses  a  non-EIDS-compatible  interactive  video  system.  Because 
EIDS  is  the  Army-wide  standard,  conversion  of  the  THREAT 
courseware  to  EIDS  compatibility  will  increase  its  utility  as 
a  training  device  and  its  potential  as  a  training  research 
instrument.  Consequently,  ARIARDA  directed  Anacapa  Sciences 
to  modify  the  THREAT  courseware  to  use  the  EIDS-ASSIST 
system . 


Project  Objectives 

The  primary  function  of  this  technical  advisory  service 
is  to  provide  EIDS  programming  support  to  ARIARDA  part-task 
training  effectiveness  researchers.  The  specific  objectives 
are  (a)  to  review  the  EIDS-ASSIST  technical  literature  and 
(b)  to  produce  a  prototype  training  program  using  the  THREAT 
videodisc,  ASSIST,  and  EIDS.  The  prototype  program  will 
emphasize  the  identification  and  discrimination  of  the  Soviet 
mapping  symbols  on  the  THREAT  videodisc. 
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Support  Provided 


After  obtaining  the  EIDS  system  documentation,  the 
Anacapa  technical  advisor  became  familiar  with  the  EIDS 
hardware:  the  system  unit,  RGB  monitor,  keyboard,  lightpen, 

joystick,  a  hard  disk  drive,  and  keypad.  Then  the  Advanced 
MITAC  IVD  trainer  was  ported  to  EIDS  from  its  original 
delivery  system  to  provide  familiarization  with  the  EIDS 
computer-videodisc  communication  interface,  input  devices, 
and  video-graphics  overlay  controller. 

All  available  ASSIST  documentation  and  software  were 
assembled  and  studied.  Preliminary  THREAT  courseware  design 
began  using  sample  ASSIST  code  as  a  model.  The  courseware 
was  developed  to  train  the  names  and  appearance  of  Soviet 
mapping  symbols  using  the  ASSIST  authoring  system.  The 
vidr  'isc  from  the  THREAT  trainer  was  used  to  provide  the 
vie.-  -  nagery  for  the  symbols  .  Computer  graphics  for 
presenting  the  introductory  and  instructional  material  were 
also  developed  for  use  in  the  courseware. 

After  completing  the  ASSIST  database,  the  program  was 
tested,  revised,  and  debugged.  The  feasibility  of  various 
options  in  courseware  design,  such  as  use  of  computer¬ 
generated  graphics  instead  of  videodisc  imagery,  was  also 
evaluated.  All  options  identified  by  ARIARDA  were  feasible 
using  the  EIDS  system. 


Support  Projected 

All  work  assigned  to  Anacapa  Sciences  by  the  original 
ARIARDA  tasking  has  been  completed.  If  additional  work  is 
required,  Anacapa  personnel  will  continue  to  support  THREAT- 
related  proaramming  of  the  EIDS  computer. 
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